Ins and Outs of Inputs

While I was working on GSTFS (from my previous post) I also managed to stumble on a bug in GStreamer. Of course at the time I didn't know it was a bug since I was GStreamer greenhorn (I actually still am :) ).

What was it about? Well when I was converting mp3 files with pipeline

[source ]! decodebin ! audioconvert ! lame  ! id3v2mux ! [output]

Id3tags were lost in conversion. This was a real bummer for me, since my player fully supports id3v2.4. When I read through the docs, man page and I could not find any solution I did what every self respecting geek would do. I fired up my IRC client and joined #gstreamer on freenode in search for help. After a few advices that didn't work we closed the issue with me filing a bug caused by unknown reason. Originally the bug seemed to be part of demuxing problem with mp3s (ogg files didn't have this problem).

After some time, I finally had a revelation. What made my mp3 files special? Nothing. Except that all of them had ReplayGain tags calculated with mp3gain. I removed the ReplayGain tags, tried the same gstreamer pipeline as before and....Voila! My converted files were not missing id3tags anymore. The bug filed in Bugzilla is not closed yet, but now the developers at least know where to look for it.

Now I have a request for all 2 people reading my blog (including me :) ). If you find a bug in opensource (or free, whatever you want to call it) software, pleeeeeaase report it. Ideally you first make damn sure you are actually not filing PEBKAC so that developers don't waste their time. If you can help with the testing of the fix. If you are not skilled enough you can always go to the IRC channel or send and email and ask if maybe documentation needs clarification or other relatively easy chores need to be done. There's always stuff to do. Just read the post 5 Ways to Contribute to Open Source Projects Without Coding.

In my previous post I mentioned project to transparently convert media files uploaded to my mp3 player. At first I wanted to create my own project from scratch. But then I searched around on the net and FUSE wiki and I found Gstreamer filesystem or GSTFS for short.

GSTFS is based on GStreamer multimedia framework to handle media conversions and FUSE to create virtual filesystem. Because of GStreamer, simple changes on command line allow you to do almost any task concerning media files. Conversion of music files, videos, resizing of pictures and more.

I started playing with GSTFS trying to convert my music collection to lower bitrate mp3s. Simple 'cp -R * music/ music_converted/' should have worked. But it didn't. Why? Well GSTFS shows non-converted files as 0-sized files. And cp tried to optimize copying by actually not copying empty files. It doesn't even try to read them. That meant running cp twice since second run would see actual sizes. Even then there is a problem with expiration of file cache so if your music collection is more than a few files, you are out of luck.

And here we come to great advantage of opensource (at least for me). Source code of GSTFS is available, so I fixed this small bug and send a few line patch to original author Bob Copeland. I also asked if he could perhaps create public repository of sources on repo.or.cz. Interestingly enough I was aparently not the only one to ask for it. And so, lo and behold, Git repository of GSTFS is online. You can now find my work on improving GSTFS in mob branch in the repository. Hopefully I will be able to contribute more to this great idea and my code will actually make it into the main branch :)

We all know that, right? Right. Who am I kidding? Most people don't notice the difference between 96kbps 4x re-encoded mp3 and FLAC. Of course it depends on your setup. Headphones, mp3 players. All of it makes your experience better (or worse).

Since I bought my first Koss Porta Pro headphones I realized that there are headphones and HEADPHONES. With some you don't even hear the important parts, while others make you realize how much noise there is in the source :). And so my music collection is now mostly flac, high quality ogg or vbr mp3. Normally I don't care about the size of music files. Storage is not that expensive these days. But I have an old Cowon U3 music player (still cannot find anything better) with only 2GB of memory. Of course that's where size comes into play. What I usually do is convert music to lower bitrates (160 kbps vbr usually) before transfering them to the player. But I don't keep those converted versions around since I don't have that much space lying around :). So I am wasting time chosing files to transfer, then converting them and finally copy them to player.

Manually converting and then transferring files is kind of a bummer though. Now I realized...I can actually program right?! So how about making a FUSE virtual filesystem on top of vfat filesystem on the player. This virtual filesystem would convert music files being copied to the filesystem to specified format in background. Processor speeds are fast enough to do this more-less realtime these days so why not?

How will this affect my workflow? Compare:

Now	Virtual FS
Copy files to temporary directory Convert big files to lower bitrates Copy files to mp3 player	Copy files to mp3 player

So far this is just an idea. I don't know of any other project doing the same thing (there are a few dealing with general data compression but not media files specific).

Expect more to come (just don't expect deadlines :) ).

*EDIT* As it happend there are already FUSE project that do exactly what I had in mind. I guess I should check the page more often. The projects are GSTFS and MP3FS where the first one seems more promising and flexible.

I am pretty sure everyone has seen CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart) before. Maybe you didn't know the (full) name but you have encountered it when registering accounts, posting comments or accessing some parts of web. You know, those annoying things that are exercising your ability to recognize distorted words with weird backgrounds.

CAPTCHAs are used to protect against automated attacks. For example automatic registration of new users on Gmail would create great opportunities for spammers. CAPTCHAs are mostly working, even when they get "hacked" from time to time. The biggest problem? They are reaching levels where even humans are having problems reading the letters. I still have nightmares when I remember CAPTCHAs used on RapidShare. Telling cats from dogs was not that easy for me somehow. I am not sure about "hackability" of reCAPTCHA, but as far as usability goes, it's one of the best ones for me. Too bad only a few sites are using it.

The main problem of CAPTCHAs is not the complexity but relay attacks and human solvers from 3rd world countries paid for solving thousands of CAPTCHAs a day. What we really need is CAPTHHA (Completely Automated Public Test to tell Humans and Humans Apart). Computer science is far from being able to tell humans with "clean" intentions from those being paid to get past the defences. One solution would be to issue certificates of "humanity" signed by central authority. You could then ban users that were misusing their certificates. There are of course privacy and security problems with this approach, not to mention financial "issues", so I guess this is not how it's gonna work.  Other approaches have also been tried, but they usually have problems with disabled people. I am certainly interested how Computer Science solves this problem.

I'm slowly starting to feel like Google Fanboy(tm), but Big G has made an interesting announcement recently. Dubbed "Clean Energy 2030", proposal tries to encourage several ways to achieve usage of "clean" energy by year 2030. I suggest you read it, especially if you like sci-fi. Basically they suggest 3 complementary things to do:

Of these, first two seem OK. But electrifying transportation? Especially in US where you cannot buy car with engine less than 2000cc? I will watch closely. I still remember those sci-fi movies that showed flying cars in year 2000 and I am still dissapointed there are almost none.

I would love to see future come sooner, possibly while I'm still alive, but I am a little bit sceptical. Google might chip-in with generous 45$ mil this year, but will goverments follow? I doubt it. Still, hope dies last. I still have this dream of Earthlings being one big nation where it doesn't really matter which part of Earth are you from. It just matters you are not from Qo'noS or Minbar. And this "cheap energy for everyone" initiative reminds me of these dreams. Oh well, one can dream.