purpose: Will Kahn-Greene's blog of Miro, PyBlosxom, Python, GNU/Linux, random content, PyBlosxom, Miro, and other projects mixed in there ad hoc, half-baked, and with a twist of lemon
[
home
|
blog home
|
recent activity
]
Page 1 of 28 >> (less recent)
Fri, 18 May 2012
elasticutils status -- May 18th, 2012
A few months ago, I "took over" maintenance of elasticutils. We use it in
SUMO as the API for building search
queries with elasticsearch.
One of the first things I did was spend some time figuring out whether
we should keep working on elasticutils at all. django-haystack also provides a django-ish API for
working with elasticsearch. Why have two libraries that at a high level
do the same thing?
The thing is that they're not exactly the same. django-haystack is really
great and supports a variety of backends for search, elasticsearch being
one of them. Right now, it only has support for elasticsearch in 2.0
which is in either an alpha or beta state now (their web-site could use
some updates). However, because it supports a bunch of backends, it only
supports functionality that works across all of them.
elasticutils, on the other hand, is elasticsearch-specific. As elasticsearch
adds functionality, we can, too. That's the compelling reason to keep working
on this library. However, django-haystack has some awesome ideas that we'd
like to implement in elasticutils, too. This will fix some sharp edges in
elasticutils, but also make it much easier for projects to switch from one
to the other.
Currently, elasticutils only handles the query side of things. django-haystack
handles that, but also has an API for defining mappings, indexing, and
all the other things you need with a search system.
Thus, Rob Hudson and I are going to
embrace and extend elasticutils to:
- fix the current situation where it seems every elasticutils user is
actually using their own branch with additional functionality in it (ew!)
- implement the rest of the things you need with a search system
- document the things we've learned while working with elasticutils because
at a minimum, it seems most of the Mozilla projects that use elasticutils
bumped into, spent time on, and solved the same problems---that's a huge
waste of time and a failure on my part
One of the things users of a library need is for the library to be a mature
project with releases, tagged version, documentation, tests, stability,
reliability, reproduceability, communication, community and all that.
Thus, I'm also going to spend some time to turn this into a real project.
Towards that end, I created #elasticutils on irc.mozilla.org where
we'll talk dirty elasticutils stuff. If we end up with more people pitching
in, we'll create a mailing list. But for now, IRC will do.
My next step is to spend a little time cleaning up what's in the master
branch, then tag and release a baseline version.
After that, I'm going to spend time identifying, thinking about and merging
in the divergent functionality in the various branches while Rob works on
continuing his imperative mapping work.
I think in a couple of months, we'll be in a better place and that'll make it
easier for Mozilla projects and anyone else who wants to use elasticutils
to use and contribute to it.
If you're a user of elasticutils, please come hang out with us! Let us know
how we can better help you.
Thu, 10 May 2012
Letting it run out
A couple of years ago I applied and was granted membership into the
GNOME Foundation. At the time, I was helping out with GNOME Journal
and running GNOME Miro Community.
Since then, GNOME Miro Community fizzled out and stopped because it didn't
find a compelling reason to exist (wasn't interesting for marketing, didn't
do a good job of indexing conference video, ...). GNOME Journal fizzled
too and I stopped working on it.
Also, in September, I left PCF where I was working on Miro which uses a lot
of GNOME technologies.
Given that, I decided not to renew my membership. I'm a little bummed, but
I'm not doing anything. I do wish them well. I will miss the periodic
anti-FSF fireworks.
Sun, 15 Apr 2012
pyvideo status: April 15th, 2012
What is pyvideo.org
pyvideo.org is an index
of Python-related videos on the Internet. For the most part, it's a
collection of videos from Python-related conferences. Saw a session you
liked and want to share it? It's likely you can find it, watch it, and
share it with pyvideo.org.
Status
I started richard in order to build
pyvideo.org. I threw it together in like 3 weeks and it had a lot of issues.
Since then, I've had a lot of help from Reiner Gerecke and the project is
moving along. There are a few more things I want before I consider richard
to have a critical mass sufficient to do a version release, but we're
pretty close. I'm pretty excited about that.
I updated pyvideo.org with the latest richard code today. I also spent some
time fixing the speaker data---it's still kind of screwy. Amongst other
things we had 3 Wesley Chuns! Now there is only one.
M.-A. Lemburg submitted two more conferences to get added for a grand total
of three in my queue now. I upstreamed some fixes to vidscraper a couple of weeks ago. Once we
land the API changes in richard, I'll be able to add new conferences much
more easily. Right now I have to do each video by hand... it's kind of a
drag.
Anyhow, things are going well, though the site looks the same and many of
the issues are still there. You'll just have to take my word for it for
now.
If you're interested in helping out, I sure could use you! Testing,
documenting, fixing the layout of some of the pages, implementing
new features, fixing project infrastructure---there are a lot of
different kinds of things you could help out with. Even looking through
the code and pointing out egregious issues---that's also very helpful
and very welcome.
Issue trackers are here:
Code is here:
Tue, 20 Mar 2012
pyvideo status: March 20th, 2012
What is pyvideo.org
pyvideo.org is an index
of Python-related videos on the Internet. For the most part, it's a
collection of videos from Python-related conferences. Saw a session you
liked and want to share it? It's likely you can find it, watch it, and
share it with pyvideo.org.
Status
I launched pyvideo.org on March 10th. That was 10 days ago. In those 10
days, I fixed a bunch of issues, screwed up the site a couple of times,
got dozens of emails many with encouragement and feature requests,
talked to Craig with Universal Subtitles,
and watched Google Analytics with some amazement. It's been a really
wild 10 days.
I think overall the experience was a successful one, but there were a
series of failures most of which were because I wasn't ready to go
live when I went live, but I had really run out of time. Lack of feeds,
media urls, easy ways to fix data, crummy experience on iPads and Android
tablets, ... The list is longish. So it goes.
Of all the successes, I'm the happiest for the following:
The site had 100,000 or so visitors. That roughly means that PyCon
2012 reached 98,000 more people than it otherwise would have. That
means that 98,000 more people heard about programming practices,
robots, outreach, libraries, applications, case studies and all the
other things we learned at PyCon. That's a big win.
Thank you to the PyCon conference crew! You run a damn good conference.
I got dozens of emails. Many with encouragement, some with feature
requests, some with data fixing requests, and some pointing out
issues with the site. I think it's awesome that so many people
took the time to email me feedback to make this resource better.
There are 8 forks on Github of richard. Further, I've had the
pleasure to add two new people to the AUTHORS file and look forward to
adding a third very soon. Thank you to Reiner, David and Alejandro!
I work on richard and pyvideo in my spare time. I think this software
is useful for other conferences and other groups which is why I made
it Free Software and kept it separate from pyvideo. I can't possibly
do everything I want to do with it, so I spent time writing documentation
to ease the barriers to entry for new contributors. I feel really good
about the work I did.
I've spent the last few days writing bug reports for things that need
to be fixed, fixing some, fixing some data, and working on feeds. I
haven't spent a ton of time on feeds, yet. I really hope to finish it
soon, though.
If you're interested in helping out, I sure could use you! Testing,
documenting, fixing the layout of some of the pages, implementing
new features, fixing project infrastructure---there are a lot of
different kinds of things you could help out with.
Issue trackers are here:
Wed, 14 Mar 2012
PyCon US 2012 videos status
What is pyvideo.org
pyvideo.org is an index
of Python-related videos on the Internet. For the most part, it's a
collection of videos from Python-related conferences. Saw a session you
liked and want to share it? It's likely you can find it, watch it, and
share it with pyvideo.org.
The plan
I just got home and I need to catch up with work. My immediate plan is to
fix the software and deployment scripts so I don't epically screw up the site
again. (I'm really sorry about that.)
After that, I want to implement feeds---that's a much requested feature (and
for good reason).
After that, I'll be fixing the myriad of other issues with the software and
the site. I'm tracking issues in the
pmc issue tracker and the
richard issue tracker ("pmc"
stands for Python Miro Community and is the github repository I've been using
for the site for a while now---now it covers pyvideo.org stuff---and "richard"
is the software that runs pyvideo.org).
Sat, 10 Mar 2012
pyvideo is live!
pyvideo.org is live!
About three weeks ago, I decided I really needed a bunch of things that
Miro Community didn't have and it was really hampering my ability to do
what I wanted to do with the site and also interoperate with Carl and
Ryan's system. It was time to write my own software and host my own site.
The software is called richard.
It's released under the AGPLv3, is currently hosted on Github, and it's
in a ridiculously alpha state. richard is at its heart a video indexing
site. I'm working on it in my "copious spare time" and hit a critial
mass last night---enough to launch pyvideo.org.
I'm pulling in PyCon US 2012 videos
as they're coming in right now.
Oh, wait... that means I should have started this blog post differently....
What is pyvideo.org
pyvideo.org is an index
of Python-related videos on the Internet. For the most part, it's a
collection of videos from Python-related conferences. Saw a session you
liked and want to share it? It's likely you can find it, watch it, and
share it with pyvideo.org.
More soon!
Mon, 30 Jan 2012
Migration to Trac: How some small projects balloon into massive projects when you have good intentions and you're not paying attention
Since around July, I've been working on migrating MediaGoblin from
Redmine hosted on the Foocorp servers to Trac hosted on our server. The
project looked small enough initially, but then suffered from a series of
complications that turned the small project into a big project that
took about 6 months and involved writing about 5000 lines of code.
Pretty crazy. I don't even want to estimate how many hours I spent on it.
Oy.
It wasn't a terrible project, though. Parts of it were educational.
I'd written scrapers in Java and Perl a long time ago, but hadn't written
anything with Python, lxml, and cssselect . I read through parts of
Trac and Trac's db schema. I also read through Trac plugin code.
The best part about it is that it's done now. This gives the GNU MediaGoblin
project more autonomy and also more flexibility for adjusting the project
to meet their specific needs. That's good stuff---time for cake!
Python Miro Community status: January 30th, 2012
Status
Kuma sent me an email pointing out that the download url for one of the
PyOhio 2010 videos was kicking up an HTTP 404. We have this problem
semi-regularly with the blip.tv urls. A while back I wrote a script to go
through a category of videos on Python Miro Community and verify that the
urls were good.
I ran the script on the PyOhio 2010 videos and 16 of 20 of them had bad
download urls. That sucks.
Between moving things around and changing their interface in a way that makes
our uses for blip.tv painful, I'd like conferences to stop using blip.tv.
Maybe YouTube is a better venue.
Amongst other things, Universal Subtitles works better with videos posted on
YouTube than blip.tv. In fact, if we switched to YouTube, it'd make it _so_
much easier to add Universal Subtitles support and I wouldn't have to use
Ogg Vorbis which doesn't work in Safari or on Apple devices.
That's all I have to say about that.
I haven't worked through my queue of conferences, yet. The queue of
oustanding tasks is maintained in the
PMC GitHub issue tracker.
I'm pushing to catch up with things so that it's in a better state for
Pycon 2012.
I'm going to PyCon 2012, though this time as a Mozillian. Looking forward to Erik's talks on
Parsing horrible things with Python
and
Speedily practical large-scale tests,
Asheesh and Jessica's talk on Diversity in practice: How the Boston Python
Meetup grew to 1000 people and over 15% women, Karen's talk on Improving Documentation with "Beginner's Mind" (or: Fixing the Django Tutorial), and others, too.
Wed, 25 Jan 2012
my standing desk
Back in October of 2010,
I created a standing desk. A friend of mine bought a Sears workbench and was using
that and while it was pretty cool looking, I wanted something I could more
easily nail things into. I've been meaning to write about this for a while.
I bought some two-by-fours and plywood and built most of it with that and some
scraps of wood I had lying around.
Since then, I built a stool to go with it and bought a shelf thing from Ikea
that goes to the right of it.
I think it took a day to build the standing desk, though I've tweaked it a
bit since I originally built it. It took a day to build my stool. Total
cost in parts for both pieces was definitely under $100.
Result is this:
This set up has lots of horizontal space, is set at the right height for
me, lets me sit when I'm weary and stand when I'm not, and has some storage
capacity so I'm not surrounded by stuff.
"Wait! What's up with all those computers?", you might ask. Well, the computer
breakdown is like this:
- one work laptop
- one laptop that I use for Skype/Vidyo for work
- one laptop I use for home stuff
- one desktop I use for home stuff and games
- one desktop that's the current PCF Windows build
box for Miro
- one Macbook that's a backup PCF OSX build
box for Miro
Then I have:
- one settop computer I'm not using at all
- one dead laptop
The dead computers should go away. The PCF build boxes will eventually be
re-homed, too.
And that's the state of my desk!
Wed, 11 Jan 2012
Fix for vagrant keys permission issue
Tim threw together a vagrant setup for Kitsune based on what
Alex did in November. I went to give it a try but I'm using vagrant
0.8.10 and it died.
Important part of the error message was:
/var/lib/gems/1.8/gems/net-ssh-2.1.4/lib/net/ssh/key_factory.rb:38:in
`read': Permission denied - /var/lib/gems/1.8/gems/vagrant-0.8.10/keys/vagrant
(Errno::EACCES)
Issue is documented at https://github.com/mitchellh/vagrant/issues/235.
My fix was to do this:
saturn /var/lib/gems/1.8/gems/vagrant-0.8.10/keys> ls -al
total 24
drwxr-xr-x 2 root root 4096 Jan 11 20:00 .
drwxr-xr-x 10 root root 4096 Jan 11 20:00 ..
-rw-r--r-- 1 root root 821 Jan 11 20:00 README.md
-rw------- 1 root root 1675 Jan 11 20:00 vagrant
-rw-r--r-- 1 root root 1464 Jan 11 20:00 vagrant.ppk
-rw-r--r-- 1 root root 409 Jan 11 20:00 vagrant.pub
saturn /var/lib/gems/1.8/gems/vagrant-0.8.10/keys> sudo chmod 644 vagrant
saturn /var/lib/gems/1.8/gems/vagrant-0.8.10/keys>
That fixed it for me. Figured I'd document it so that I could find it again
when I bumped into it again.
Page 1 of 28 >> (less recent)
pyblosxom::1.5-dev git-master
Copyright 1996 to 2012, Will Guaraldi Kahn-Greene, under the Creative Commons BY-SA 3.0 license
Will's Blog by William Kahn-Greene is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
![[Valid RSS]](/~willg/images/valid-rss.png "Validate my RSS feed"){kind=small}
![[FSF Associate Member]](http://bluesock.org/~willg/images/fsf_member_button.png){kind=small}
