UPDATE: Because of the wonderful reception across the internet, I’ve put together an instruction page on how to get this set up on your own server.

I don’t really like database driven photo management software, and prefer instead to manage my photos in a good old no-nonsense directory structure. For this reason, I was particularly attracted to Zenphoto as a means of getting my photos online, as it works on directory structures. Unfortunately, Zenphoto is horrible; it’s riddled with bugs, inconstant, a cluttered architecture, and most of all, it’s extremely slow. Every time it runs, it re-scans directories and makes a bazillion SQL calls. The viewer interface is also outdated and clunky, having a different html page for each photo. So I went back to the drawing board and considered how to make things better.

Introducing PhotoFloat. The idea is this — instead of scanning and caching metadata and thumbnails during page load time, everything is to be done prior. It’s a bit of an old school mentality. There is a script that generates static json files of metadata and album structures and static thumbnails of images, so that all the content can be served directly by Apache. Why? Because I only need to generate new thumbnails and data files when I upload new images (or alternatively, on a cron job). So that’s what I did; I wrote a simple python script that walks a directory structure looking for new or changed images and albums. It’s smart too — to be super zippy, it does file modification time comparisons. It also cleans up after itself, deleting stale files.

So I have all my original images on my webserver, because I have Dreamhost’s unlimited hosting. I also have another directory that I populate with symlinks to the directories I actually want online. Every time there are new images, my python script fires up, and updated json data files and thumbnail files are generated.

Great, but where does this leave us? What can we do with json files? This is where things become wonderful. Since all the data for the gallery is AJAX fetchable, there is a single html page and a single javascript file that takes care of the whole gallery. That’s right — all of the display of views is done client side, and in one page load.

To keep track of pages and for swapping around links, each different album and different image has it’s own hash url, like, for example: #!/new_hampshire_in_snow_3.15.11-3.17.11/img_1919.jpg. It’s all lower case with naughty characters stripped out to keep up with the patterns of wordpress and other web apps. These function as permalinks.

The albums have extensive support for EXIF metadata, which can be loaded by clicking ‘show metadata’, and a transparent box slides up over the photo. There’s also the ability to download the original photos.

Each album gets a randomized thumbnail which assigns probabilities to each image in the album based on the number of images in each album and the depth of subalbums. The randomization algorithm is all done at client side.

Images are preloaded. Album data is prefetched. Everything is cached sanely. JSON files are gzipped. There are animations between views and smooth scrolling. The right and left arrow keys work. Clicking on the photo advances it, like on Facebook. Finally, I do include one dynamic script — a simple php script that takes old Zenphoto URLs and translates them into the new ones, so that people with old links can still access the same photos.

Essentially, there are a lot of little details that had to be done right, and to my knowledge, no web gallery that works on directory structures has done it well, making an ajaxy and speedy gallery. So now you have PhotoFloat. I’ve just finished writing it, and the code is a bit of a mess, but let me know if you have any suggestions or find any bugs.

You can browse the code in the git repository or try it out live on my photo site. If you make any modifications of my code or use it on your own site, please inform me and send any modifications back to me. Remember to run make on the web directory to minify the css and javascript, and also, be sure to change the google analytics tracking ID in web/js/999-googletracker.js.

Comments? Suggestions?

Update 2 for KDEers: It looks like some people from kipi-plugins and kphotoalbum are interested in building integration for this in.

Update 3: Following a suggestion in the comments below, URLs now use #!, which google translates to a special query string, and I’ve written a serverside component that executes the JavaScript and displays static content for googlebot. This allows the metadata to be crawled.

May 8, 2011 · [Print]

54 Comments to “PhotoFloat — A Web 2.0 Photo Gallery Done Right via Static JSON & Dynamic Javascript”

  1. GSinnott says:

    Niiice!

  2. Bookmarked! Definitely to be revisited and attempted for my own photos to be put online.

  3. Petr Viktorin says:

    So without Javascript, your gallery degrades to a blank page :(

  4. WMonceaux says:

    Nice gallery software. It definitely fills a void. Also, great photos on your site!

  5. evergreenpsyche says:

    So, any thoughts on possibly segregating the frontend and turning it into an export plugin using kipi-plugins? Or is this something that’s deceptively more complicated than it seems?

    This sure beats the pants off of what it currently uses for gallery export

    • This very much would be possible. I’m already in contact with the fella from KPhotoAlbum about his ideas. I’d love to get this kipi-integrated. Frontend and backend are already separated. The frontend works by simply looking for json files and thumbnails in a certain directory. Nothing more than that. So it’d be easy to make many different generators of those json files and thumbnails to integrate. My python scanner script is only one such possibility.

      • evergreenpsyche says:

        Very cool. I don’t know if kipi-plugins has any sort of restrictions on how complex or detailed the configuration UI for the plugin can be.

        Particularly in something like this, there are a plethora of options that could be offered; fonts, layout, image compression, offer “download original,” etc etc.

        I like most of the design choices you’ve made for the frontend, but there are some things I’d like to be able to change. Don’t know how I feel about the italicized font in the overview, and I would also like to play with horizontal padding between albums.

        A thought that occurred to me while writing that; different users’ preferences for visual design could probably be placated by offering 3 or 4 default packaged themes for the gallery.

        On that note, how easily themeable is this? Is the layout code mostly in javascript or CSS?

        great job, btw, way to scratch that itch :)

        • The vast majority of the layout is done in CSS, so it would be fairly trivial to just supply additional CSS files.

          I’m really no web designer and I haven’t much of a clue what I’m doing w.r.t. web design, so if you have any suggestions about the current theme, do let me know. Spacing, colors, fonts… I REALLY need some feedback in this department.

          I imagine things could be configurable with a config.json, but this is an extra asset that has to be patched. Maybe the best way to offer a feature like “download original” would be to in the javascript check whether the html element exists, and if it does, supply the link, and if it doesn’t, don’t. That way, to disable it, all you’d have to do is remove the relevant part from the html. What do you think?

  6. evergreenpsyche says:

    I’m really no designer either, I just know what I personally prefer in looks — but dare I say, this might qualify for a Season of KDE project? ,:) http://toscalix.blogspot.com/2011/05/season-of-kde-we-need-you-kde.html

    Regarding “download original,” i’ll have to skim the javascript to see how you have the code laid out before answering that, and I’m on a phone right now, not a proper computer. */bookmarks for later tonight/*

  7. Elazar Leibovich says:

    Use google hash-bang urls so that it’ll be able to index the meta-data of the image

  8. Elazar Leibovich says:

    Forgot to mention the url
    http://code.google.com/web/ajaxcrawling/

  9. jbd says:

    That’s a great application ! Is there a way to directly draw the original image instead of the thumbnail when its dimensions are less than the precomputed images ?

  10. [...] not quite. A comment in my blog post pointed me toward Google’s AJAX Crawl specification, which is incredible. Basically, sites [...]

  11. ErMejo says:

    I just downloaded an tried your software.

    So far I am positively impressed: the concepts/reasoning behind this project match what I expect and what I did not find in other ‘gallery’ tools.

    Some remarks (I guess most of these remarks are caused by the youth of the project):

    - error handling: I got an error on one of my images while converting ” File “/usr/lib/python2.6/dist-packages/PIL/PngImagePlugin.py”, line 337, in load_prepare
    raise IOError(“cannot read interlaced PNG files”)” This is clearly a problem of PIL, but handling the exception gracefully (maybe log it and instead of the image put a warning) would be ideal.

    - sorting: it would be great if the user could control the sorting of directory and files. E.g. in my case sort the directories DESCENDING and files ASCENDING (maybe everything needed is a two line change in PhotoAlbum.py, replacing the sort() calls with sort(reverse=True)

    • About error handling: I had fixed this last night for somebody else, I think, who had a fairly similar problem with this commit http://git.zx2c4.com/PhotoFloat/commit/?id=6d75a3ed642529a99227d578f74ec14bd7e03d42 . Be sure you have the latest. If this doesn’t fix it, would you send me a full stack trace of the error and the PNG you used to trigger it?

      Sorting — yes indeed, it needs some more customization. I think I’m going to move to model where you can specify a config.json with various values to give to the python sorter, which can include separate settings for different directories. Additionally, each directory will be able to have it’s own config.json for overrides. This will be combined with an equally as configurable system for the javascript application (such as disabling ‘download original’). Stand by.

      • ErMejo says:

        I downloaded the newest version from git and there is a bug:

        File “/home/lombardo/t/PhotoFloat/scanner/PhotoAlbum.py”, line 189, in _metadata
        self._attributes["meteringMode"] = self._metadata.metering_list[exif["MeteringMode"]]
        IndexError: list index out of range

        Notice the different upper/lower/cameling of MeteringMode.

  12. Angel Blue01 says:

    This looks to be at an awesome start! A feature that’s very important to me in a gallery is the ability to easily order prints of the originals from a service such as Shutterfly or Snapfish. Would it be possible to integrate that in a future version?

    • Jason says:

      It would be trivially possible.

      I have no interest in doing it.

      PhotoFloat aims to be clutter free.

  13. mxttie says:

    hi Jason,

    another great soft ;)
    I was wondering: are you still satisfied with dreamhost? would you recommend it?

    matthias

  14. Hey, you’re a beast. Very inspiration coding.

    I’m developing an exciting social networking site called Vines and we are testing your software out. One question:

    What do you think the best way to add user comments to the photos?

    • Jason says:

      Wonderful to hear!

      Because comments are dynamic and potentially lenghty, you can’t really load them statically with the rest of the photos. Instead, I think it’s best to preload all the image data, like PhotoFloat does, but then make a separate AJAX request when each photo shows. Usually someone looks at the photo for a few moments before deciding to read/write comments anyway. This way, the data is always fresh, even if they’ve left open the window for a while and flip around the photos later in the day.

      So, you do things at first like PhotoFloat does, but then on each photo load, another AJAX request is made to fetch the latest comments and insert them in a div at the bottom/top/side of the photo. This is, as far as I can tell, what Facebook does in their latest photo viewer iteration.

  15. Martin says:

    Soon after found this software and sow all your projects I can say you’re becoming my idol man :) This is the best ever photo album I ever wanted. It’s actually something I have been looking for form months. I admire your work. I am usually not sharing my photos to any social network sites so this is what I need for the blog of my little baby and I am going to post his phptos soon using PhotoFloat.

    I have one suggestion for a feature that would be nice to have. Actually I am willing to start learning python so if you are not going to implement it I may try by myself, but the problem I have lack of free time.

    You know that on modern job sized photo cameras there is usually possibility for recording a video as well. Those days I am keeping my short video clips together with photos and locally I manage them with Shotwell application. It would be nice I think if PhotoFloat is also supporting the cool video tags in html5 and capable of generating simple thumbnails and putting videos just among the photos. Whoever want’s to play them just click on play button otherwise they should pass just like photo on the row. It’s not necessary to have flash player fallback in case browser is not html5 capable I think. Sooner or later there would be no more such browsers. The video transcoding again can be done automatically by the script using ffmpg etc. there must be som python module suitable for this.

    Let me know what you think .. I guess it would be wonderful feature … everyone is taking short video clips from time to time it’s fun. :)

    All the best!

  16. Martin says:

    Hi,

    I got this error when I ran main.py to update the cache:
    ./main.py ../web/albums ../web/cache
    2012-06-22T15:05:22.771311 [walking] albums
    Traceback (most recent call last):
    File “./main.py”, line 18, in
    main()
    File “./main.py”, line 12, in main
    TreeWalker(argv[1], argv[2])
    File “/home/martian/mafiainc.net/Дневника на Камен/PhotoFloat – Kamache/scanner/TreeWalker.py”, line 16, in __init__
    self.walk(self.album_path)
    File “/home/martian/mafiainc.net/Дневника на Камен/PhotoFloat – Kamache/scanner/TreeWalker.py”, line 53, in walk
    entry = os.path.join(path, entry)
    File “/usr/lib/python2.6/posixpath.py”, line 70, in join
    path += ‘/’ + b
    UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xd0 in position 27: ordinal not in range(128)

  17. Florent says:

    Hi !
    Great work, this is exactly what I was looking for.
    However, my cron job was unable to run the program without a python exception on file names encoding during tree scanning. Almost the same as previous post but this time in files being scanned, not initial paths. And my special characters are french accentuated letters, not cyrillic.

    Everyting ran smoothly when run with my user as my LANG variable was set to en_US.UTF-8 but couldn’t solve the issue when setting this value in the cron job script… I couldn’t figure out why…

    Anyway, I fixed it by adding this to main.py :

    import sys
    reload(sys)
    sys.setdefaultencoding(‘utf-8′)

  18. Antonio says:

    Hello, your gallery is really great – fast, slick, and dead easy to setup. The only problem I have is that the JS code apparently breaks on IE7 – a friend reports me she can’t see my gallery from her oldish browser at work (she cannot upgrade it, of course). I’ll give it a go at fixing it as soon as I find an IE7 to test with – in the meanwhile, thanks for the great work.

  19. Antonio says:

    Hello again,
    solved the IE7 problem: the Javascript engine does not allow to access strings as arrays. Here is a working patch:

    http://pastebin.com/CwJzXsYP

    Great work, really!

  20. Kostya says:

    Hi, is there any possibility to add security features? I don’t want to all people see my private library. Thanks?

  21. Kostya says:

    It works well man. Maybe you wish to move the code to github? So other people could easily participate in improving it?
    It would be nice to have possibility to restrict the library for special users. Any suggestion? (apart of simple restriction by apache/nginx configuration). Thanks!

  22. Sebastian says:

    Hey together,

    very nice project, which I tried to run on my RaspberryPi. Unfortuounatly I always got the screen “Forgot my Camera” while running a nigix server.
    So I had a look into the htacess files and converted them to nigix. But I’m still suffering this problems.
    Do you guys have any hints for my what the problem might be?

    Greetings from Munich,
    Sebastian

  23. nodiscc says:

    Hi, this looks like a nice project. I have not been able to run it and posted details about my problem to the mailing list. @Jason could you have a look? Thanks, looking forward to ditch Zenphoto

  24. woodchuck says:

    It’s genuinely very complicated in this active life to listen news on Television, therefore I only use world wide web for that purpose,
    and get the newest information.

  25. home says:

    Pretty! This has been an extremely wonderful post. Thanks for providing this
    info.

  26. Heya! I’m at work surfing around your blog from my new apple iphone!
    Just wanted to say I love reading your blog and look forward to all your posts!
    Keep up the great work!

  27. Wilbert says:

    An intriguing discussion is worth comment. I do think that you should publish more about this subject, it may
    not be a taboo subject but generally people don’t talk about these subjects.
    To the next! Many thanks!!

  28. kids clothing catalogues

    PhotoFloat – A Web 2.0 Photo Gallery Done Right via Static JSON & Dynamic Javascript | Nerdling Sapple

  29. escort seo says:

    I do consider all of the concepts you have presented for
    your post. They’re really convincing and will definitely work.

    Nonetheless, the posts are too brief for novices. May just you please prolong them a bit from next time?
    Thank you for the post.

  30. Baby Clothing stores san Francisco

    PhotoFloat – A Web 2.0 Photo Gallery Done Right via Static JSON & Dynamic Javascript | Nerdling Sapple

  31. more says:

    more

    PhotoFloat – A Web 2.0 Photo Gallery Done Right via Static JSON & Dynamic Javascript | Nerdling Sapple

  32. Yoursafariguide.com

    PhotoFloat – A Web 2.0 Photo Gallery Done Right via Static JSON & Dynamic Javascript | Nerdling Sapple

  33. Hack gratuit says:

    Hello just wanted to give you a quick heads up and let you know a few of the pictures aren’t
    loading correctly. I’m not sure why but I think its a linking issue.
    I’ve tried it in two different internet browsers and both show the same outcome.

  34. justproveit.net

    PhotoFloat – A Web 2.0 Photo Gallery Done Right via Static JSON & Dynamic Javascript | Nerdling Sapple

  35. test.sciencefa.com

    PhotoFloat – A Web 2.0 Photo Gallery Done Right via Static JSON & Dynamic Javascript | Nerdling Sapple

  36. Thank you for sharing your thoughts. I truly appreciate your efforts and I am waiting for your further write ups thank you once again.

Leave a Reply