Fraser Speirs Cocoa and Photos

Numbers on OS and hardware combinations from FlickrExport.

FlickrExport 3 contains a software update module which can anonymously report some configuration details to my server when it checks for updates. I recently did a first analysis of the numbers and here are some basic results.

Firstly, let me state the limitations of this work. I took all the software update hits from my server log and eliminated those without system information. This left about 35,000 hits. The major caveat is that I have no way of uniquing these hits. I am definitely double-counting individuals. However, FlickrExport only hits the server when it runs so I am at least counting active uses of the plugin.

Operating System

The main thing I was trying to discover is whether I should still care about Tiger. FlickrExport is nearly five years old and has a lot of Panther-era code inside it. It’s code that works, but it’s not always using the latest APIs and language features.

The results were not terribly surprising, it must be said, but it’s good to have actual data.

Architecture

Apple has been shipping Intel for three and a half years now. I no longer have access to a working PowerPC Mac, not even by borrowing one from someone.

The transition really is over, isn’t it? The next thing I was interested in was the split of hardware architecture by OS version. Were these Tiger users mainly using Tiger because they haven’t upgraded their machine?

I think that’s a reasonable conclusion to come to. Almost nobody has upgraded their PowerPC machine to Leopard. Remember that the size of these pies is not equal: the left pie is 6% of all hits, the right is 94%. The number of hits from an Intel/Tiger machine is about the same as the number of hits from a PowerPC/Leopard machine.

Models and Form Factors

More for curiosity than to actually inform what I’m going to do, I took a look at the split across the various model groups in the Apple lineup:

The poor old MacBook Air doesn’t make much of a show, but the MacBook Pro is the choice of photographers, I guess. No hits from the Mac Tablet, so I guess that confirms(!!!111one) that the new Tablet runs iPhone OS. I think.

Mac vs. Mac

Last comparison for now: form factor. It kind of falls out from the last analysis, but here’s the overall split between laptop and desktop:

Conclusion

In terms of validity, I think these numbers are at least representative. They might be slightly exaggerating the popularity of Leopard, but I don’t think they do so in a way that would change the conclusions that you draw.

Compared to the Omni Group’s update stats, my data shows greater adoption of Leopard. Omni’s stats average 78% across all applications, or 83% if you exclude the historically anomalous OmniOutliner.

The obvious conclusion is that if you’re limiting yourself or your applications to support Tiger or PowerPC, then you should stop at once. Or, at the very least, at your next major revision.

The other, less obvious, conclusion is about hardware capabilities. Just remember that when Apple stands up at WWDC and prefixes the latest and greatest with “well, we’ve got 8 cores and 16GB of RAM here so…”, just remember that the majority of your users likely have about a quarter of each.

Making Changes

So I’m coming up for air after the biggest Connected Flow news in recent times - the acquisition of Changes.app. Ian Baird and I haven’t been working on the transition long. It’s been less than a month since the idea was first floated, and now it’s done.

Some people might wonder why I was interested in acquiring a tool that was so distinct from my usual internet/photography software niche. Well, that’s just the answer: time for some diversification and something new. I felt it was important in terms of my own development as a programmer.

I felt it was doubly important as part of the long-term strategy for Connected Flow. No song stays number one forever and, whilst I have ridden the wave alongside Flickr for five years now, times always change. That’s our industry. Change is what we do.

Let me be crystal clear: I have no knowledge or reason to believe that Flickr is going away. I don’t even think it’s going downhill, particularly when you look at some of the work that Aaron is producing in the geo space. At the end of the day, a repository of three billion photos simply does not disappear from the internet overnight.

Having said that I, like everyone else, have watched Yahoo being circled by predators for a year or more. I’ve watched important people across the company be let go with varying degrees of voluntarism involved. The biggest mistake any company can make is to become wedded to their first success (look at Microsoft, for example), and I didn’t want that to happen to Connected Flow.

FlickrExport has been and continues to be a great success. I don’t want to change that one bit, but I do want to survive the day that Flickr’s star starts to grow cold.

So, Changes. Changes is the app I liked so much that I purchased the IP. I’ve been using Changes as my Git diff tool since it first shipped in late 2007. It beats the pants off looking at raw diff(1) output, coloured or not.

One of the things I like most about Changes is that there’s a visual consistency in the diff: every line corresponds to the line that’s horizontally aligned with it. Other diff tools, such as Apple’s FileMerge, try to show insertions and deletions as chunks shrinking to nothing on the other side of the comparison. Changes keeps lines aligned and that really helps visual recognition.

Changes also has a “differences only” mode for looking at diffs. Instead of the whole file, it can show you only the changed chunks. This is really powerful when you’re using Changes as an SCM diff tool.

There’s tons more, including an entire file/folder differencing engine, and I’ll write about that another day. Suffice to say that, for now, I’m delighted to be the new owner of Changes and I can’t wait to wrap up the transition work and get into maintaining and improving the app for some time to come.

If you want to try Changes, you can grab it at its new home: http://connectedflow.com/changes.

Understanding Git Submodules

I’ve been using Git full-time for over a year now, but I had not yet adopted Git’s submodule feature for my projects. Git submodules are functionally similar to Subversion’s svn:externals mechanism, but submodules do appear slightly alien and confusing at first (and second) glance.

So I went deep and here, for the internet, is my best explanation: Git Submodules are basically the same as svn:externals, except that Git submodules are locked to a specific revision and don’t automatically track the external project’s HEAD.

Git submodules behave more like svn:externals that are managed by Piston than by Subversion’s default externals.

In my experience, to understand Git, you have to understand its implementation. Git is very driven by its model layer and, once you understand the model layer, I find that everything else follows quite logically.

As you may know, Git stores commits as blobs of data and trees which describe the layout of that data in the filesystem. The commit ID is the SHA-1 hash of the blob’s contents. I simplify slightly, but that’s the core. Keep this in mind.

Git submodules are implemented using two moving parts: the .gitmodules file and a special kind of tree object. These together triangulate a specific revision of a specific repository which is checked out into a specific location in your project.

The submodules file contains two parts:

[submodule "FooKit"]
	path = FooKit
	url = git@github.com:fspeirs/fookit.git

The submodule’s definition contains a path, which is the location in your repository where the submodule should be placed. The `url` is the URL of the repository to clone from. This example is a GitHub URL but it could equally be a path to a repository on your system. Thus far, Git knows where to get your submodule and where to put it.

The second question is which commit should be checked out into the submodule’s path. You tell Git this by adding the submodule path to your index and committing.

Let’s try an example. This is a repository which contains a Git repository called “a” and another called “super”. We will add “a” as a submodule of “super”:

[/tmp/git]$ ls -l
total 0
drwxr-xr-x  4 fspeirs  wheel  136 May 11 11:03 a
drwxr-xr-x  4 fspeirs  wheel  136 May 11 11:03 super

The first thing to do is run “git submodule add” in super:

[/tmp/git/super(master)]$ git submodule add /tmp/git/a ProjectA
Initialized empty Git repository in /private/tmp/git/super/ProjectA/.git/

[/tmp/git/super(master)]$ git submodule status
-85ab8ba4edf9168ab051ded7ddbbe20861b71528 ProjectA

[/tmp/git/super(master)]$ ls ProjectA/
a.txt

Having done that, let’s look at the impact of that command on the project “super”:

[/tmp/git/super(master)]$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD ..." to unstage)
#
#	new file:   .gitmodules
#	new file:   ProjectA
#

We have the new .gitmodules file, which should be checked in, and a new file called “ProjectA”, which is the “path” of our submodule. Let’s commit these two now:

[/tmp/git/super(master)]$ git commit -m "added submodule"
[master]: created ffba648: "added submodule"
 2 files changed, 4 insertions(+), 0 deletions(-)
 create mode 100644 .gitmodules
 create mode 160000 ProjectA

Note the mode “160000″ on ProjectA - that’s a special mode for a certain kind of entry in the Git index. It’s different from normal files.

Now, if we look at the contents of the Git index, we’ll see the SHA-1 for the tracked files:

[/tmp/git/super(master)]$ git ls-files --stage
100644 831cdc0dc1b88e69aa9943cf09907ae1bcd031fc 0	.gitmodules
160000 85ab8ba4edf9168ab051ded7ddbbe20861b71528 0	ProjectA
100644 16f5c2d3aa9656fc424352e4cfaa2523c809778b 0	super.txt

Notice the SHA for ProjectA - 85ab8ba - this is the SHA-1 of the commit to which the submodule is locked in Project A. Commit 85ab8ba does not exist in the “super” repository - it refers to a commit in a submodule repository.

So Git now knows the three things required to set up your submodules when cloning a project:

• The what comes from the “URL” property in the submodule’s entry in your .gitmodules file.
• The where comes from the corresponding “Path” entry in .gitmodules.
• The when, if you will, comes from the SHA-1 stored in the superproject’s index file for the remote.

Working in a Submodule

The checked out submodule is, of course, a full Git repository in itself and you should treat it that way. It is perfectly possible to make changes in your checked-out submodule. As you commit in your submodule, the SHA-1 of the submodule’s HEAD will advance away from the SHA-1 that the superproject has stored in its index.

To return to the example, suppose some change is made in ProjectA:

[/tmp/git/super(master)]$ cd ProjectA/
[/tmp/git/super/ProjectA(master)]$ echo "b" >> a.txt
[/tmp/git/super/ProjectA(master)]$ git commit -a -m "Added B"
[master]: created 82b6450: "Added B"
 1 files changed, 1 insertions(+), 0 deletions(-)
[/tmp/git/super/ProjectA(master)]$ cd ..
Submodule 'ProjectA' (/tmp/git/a) registered for path 'ProjectA'
[/tmp/git/super(master)]$ git submodule status
+82b64501654dca53ba570827d8d3e7d465abbae5 ProjectA (heads/master)
[/tmp/git/super(master)]$ git ls-files --stage | grep ProjectA
160000 85ab8ba4edf9168ab051ded7ddbbe20861b71528 0	ProjectA

Notice that, now, the SHA-1 of the submodule’s head is at 82b6450, whilst the superproject is expecting 85ab8ba4. There are two ways Git shows you that you’re out of sync:

• “git submodule status” will show a “+” in front of the SHA-1 of the HEAD of any submodule that has advanced from the SHA-1 stored in the superproject.
• Running “git status” in the superproject will show the submodule as modified.

If you want to commit the superproject to using the new HEAD of the submodule, simply add and commit the submodule’s directory as you would any other file:

[/tmp/git/super(master)]$ git submodule status
+82b64501654dca53ba570827d8d3e7d465abbae5 ProjectA (heads/master)
[/tmp/git/super(master)]$ git add ProjectA
[/tmp/git/super(master)]$ git commit -m "Advanced ProjectA to new HEAD"
[master]: created 37750a6: "Advanced ProjectA to new HEAD"
 1 files changed, 1 insertions(+), 1 deletions(-)
[/tmp/git/super(master)]$ git ls-files --stage
100644 831cdc0dc1b88e69aa9943cf09907ae1bcd031fc 0	.gitmodules
160000 82b64501654dca53ba570827d8d3e7d465abbae5 0	ProjectA
100644 16f5c2d3aa9656fc424352e4cfaa2523c809778b 0	super.txt

Notice how ‘git ls-files –stage’ and ‘git submodule status’ now show the same SHA-1 for ProjectA?

Gotcha’s for those used to svn:externals

The big thing to remember is that, unlike svn:externals, updating your superproject from a master repository does not do the same for the project’s submodules. If you think about it, this makes sense: the submodules are locked to specific commits in their respective repositories.

It’s also important to remember the distributed nature of what you’re doing. If you advance HEAD in a submodule, then update the superproject, it’s important to remember to push submodule changes before you push the superproject changes. If you don’t, your superproject will contain references to commits that only exist in your local clone of the subproject.

Wrapping Up

This post does not attempt to cover every command for working with Git submodules. In particular, you should be aware of the ‘git submodule init’ and ‘git submodule update’ subcommands - read the man page for that.

Git submodules really aren’t that complex or scary. They have comparatively few moving parts and, to my mind, enforce a certain welcome stability and discipline in your use of external projects.

NSConference, Day One

Just finished a long and excellent day at NSConference. I would be tempted to say that all the speakers today were brilliant, but that would be immodest since I was one of them. All the other speakers were brilliant.

Matt Legend Gemmell kicked off the conference with a very measured, calm and reassuring walk through the process of creating custom controls and UI.

I, Fraser Mildly Average Speirs, followed with a tour of the photography ecosystem on Mac OS X. I was really happy with my talk. It was clear that the five-plus hours of rehearsal that I did helped me get it all across in good time. I leave it to others to judge whether the material was well-received, but I felt I did a good job.

Philippe Mougin showed us F-Script. I like, I think, many developers have known about F-Script for ages but haven’t really had the a-ha moment with it. Philippe wowed the audience by using F-Script to mess around with the objects in the Keynote presentation that he was giving. Very impressive.

My friend André Pang is a frighteningly smart chap. It’s bad enough knowing intellectually how smart he is but sometimes when he shows it off…yikes. Pure functional programming, lambda calculus and casual references to Gödel, Escher, Bach - André had it all in there.

By the time André had finished, well, I was pretty much finished too. Coming down from too little sleep, too much talking and the adrenalin of doing my presentation, I had to skip Mike Lee’s Pimp My App session and catch an hour’s sleep. Mike is never not good to listen to.

We then reconvened to record a Mac Developer Roundtable Live which, despite the name, involved at least 40 minutes of iPhone discussion. I love doing that kind of thing. I should say I love hearing from other people, but I really love giving my own opinion about stuff.

Then we ate, then we talked and talked and talked and…it’s midnight. More tomorrow, but I already look forward to NSConference 2010.

Git pre-commit hooks and the Clang Static Analyzer

For the solo developer, it’s easy to kid yourself into believing that you’ll run those tests “when appropriate”. It never really happens, does it?

I’ve always liked the Clang Static Analyzer as a tool for finding code oversights, but I’ve always found it a bit annoying to run. I always forgot the options, or something didn’t quite work as expected. That’s really my fault, rather than CSA’s.

Anyway, it occurred to me that I could use my tools to keep myself more honest about regularly running the static analyzer. I use Git for source control and Git provides a pre-commit hook feature. The idea of pre-commit hooks is that you get to run a script before the commit happens. Depending on the result code of the script, the commit will either proceed or be aborted.

I wrote a wrapper around the scan-build tool, so that I could run the analyzer by hand with my preferred options at any time:

#!/bin/sh
~/Scripts/clang/latest/scan-build --status-bugs \
	-warn-objc-missing-dealloc \
	xcodebuild -sdk iphonesimulator2.2.1 -configuration Debug

The --status-bugs flag is the trick here: it makes scan-build return a non-zero status code if it detects bugs. That’s what we want with Git pre-commit hooks: a non-zero status indicates a possible bug, and that causes the Git commit to be aborted.

Also, I think I’m right in saying that you can’t run the static analyzer on ARM code, hence the use of the iPhone Simulator SDK.

The pre-commit hook is a script at .git/hooks/pre-commit in your repository. There’s an example file in every .git/hooks that you can copy and edit. Because I wrote my wrapper script above, the pre-commit is trivial:

[(master)]$ cat .git/hooks/pre-commit
#!/bin/sh
./clang.sh

The working directory for the pre-commit script is the parent directory of your .git directory.

[Update: Peter Hosey tells you how to do this for Mercurial]

Get Out of Your Subculture

I just finished photographing the Scotland on Rails conference, held this weekend in Edinburgh. It was a fantastic three days of Ruby and Rails discussions.

Dave Thomas of the Pragmatic Programmers

You might be wondering why I, a Mac and iPhone developer, am spending time at a Ruby conference. Well, reason number one is that my friend Alan Francis runs the conference and tapped me to take some photos in return for a free ticket.

Reason number two is that it’s my annual get-out-of-the-Cocoa-subculture weekend, and I’m finding that increasingly refreshing. Not that there’s anything wrong with the Cocoa community - far from it.

I just find that I’m at a place in my programming career right now that I want to become a better programmer. I don’t particularly want to hear about how to use specific Mac OS X APIs - I can pick that stuff up in an afternoon, usually. I want to hear about software engineering practices, design ideas, experiences and problems and that kind of learning comes from anywhere. The language and the APIs are mostly irrelevant to that kind of knowledge. I learned a few new things, and came away with a ton of things to think about.

What interests me a lot at Scotland on Rails are the cultural differences from my own tribe. At SoR, you see TextMate on-projector all the time. Git is the de facto standard SCM, and Rubyists share their GitHub handles like Mac developers share their Twitter handles.

Testing is de rigeur in the Rails world. In the Mac world, some of us do it, some don’t, and all of us struggle with some aspects of automatically testing GUI applications. Every year I come away from Scotland on Rails with some new insight about software testing.

What they don’t talk a lot about in the Ruby world but which we, as programmers, obsess over in the Mac world is interaction design. About the most I heard about UX at Scotland on Rails was “if you’re not good at interaction design, find someone who is” (I freely admit I didn’t hear every word of every talk). I got the feeling that the division between visual/interaction design and development in the Ruby community is much sharper than it is in the Mac community.

I wonder if that explains the prevalence of unit testing in the Ruby community? I doubt anyone ever wrote a unit test to see if their Rails app would accept a track dragged from a table view in iTunes.

I just got home and haven’t finished unpacking my bag yet. These are just half-formed but enthusiastic thoughts on my weekend. To my fellow Mac developers, think about going to an ‘irrelevant’ conference. Come to Scotland on Rails next year. You’ll leave with something to think about, for sure.

The Devalued Prime Minister of a Devalued Government

An absolutely brilliant demolition of Gordon Brown in three minutes by Conservative MEP Daniel Hannan is doing the rounds on the internet today.

I’m happy to do my bit to spread the virus. Not only because I agree with every word of it - and I do - but the sheer economy of language, the tautness and energy in Hannan’s speech is a joy to listen to.

David Cameron should make every effort to get Hannan out of the European Parliament and into, at least, a junior position in the Cabinet come the next election. A Willam Hague for the Internet era.

Oh, and don’t try and find this gem on the BBC sites. Despite its pointedness and the prestigious venue for Hannan’s critique, I can’t find this clip on news.bbc.co.uk. I suppose it’s not as newsworthy as, say, idle speculation about a riot that may or may not happen at some future date.

iPhone OS 3.0 Wishlist

Tomorrow, Apple will be showing a sneak peek at iPhone OS 3.0. I have a few hopes, both as a developer and as a user.

As a Developer, I really want to see:

• UIMapView - the internals of the Maps application made available to all developers. There are significant licensing issues around this - Google only allows free use of the Maps API to free-to-access websites. If Apple could work something out with Google for its developers, that would be huge.
• Enhancements to the interapplication URL launching system. Right now, I can’t determine whether a URL will be handled before launching it, which is a poor user experience.
• I’d like to see some way in which applications using UITabBarController can also have a toolbar. Currently, many apps abuse the notion of perceived stability to dynamically hide and show a UITabBarController. I’d like to see some new UI element to help this situation.
• Provide a sync API to the Mac. Requiring every developer to build their own sync mechanism is an insane drain on collective resources.
• Implement styled text rendering without requiring UIWebView.
• Provide API for developers to lock out the Home button. This is important for games for very young children (who just love the clicky clicky).
• Make the camera and the photo roll easier to deal with. Preserve EXIF data.

I hope Apple doesn’t waste resources implementing a Dashboard-esque HTML5+JavaScript API. I don’t see that it’s needed, except to one-up Palm.

As an iPhone user:

• Fix springboard. The current design of pages-of-apps simply doesn’t scale. I argue for one page of icons, for the most valuable apps, and a second page of drill-down navigation into user-defined groups of apps. I don’t think there’s anything I care about more than this from a user’s point of view.
• Copy and paste. It’s been an interesting experiment to see how long we could live without this staple feature, but that experiment has failed. Implement it.
• Make screen rotation happen faster.
• Make the lock screen show more information, such as appointments, missed calls, last few emails. Also provide API for this.
• Let the user decide which app is launched with a double-tap of the Home button. I want it to launch the incredibly awesome Favorites, not Phone.

It’s worth noting, too, that Apple have made some good improvements to the App Store recently. In particular, reviews are now tied to specific versions of the software and apps have “all time” and “this version” ratings.

I remain disappointed that the approval process continues to appear abitrary and capricious, but it is at least starting to become sufficiently clear where the ’safe zone’ is. That safe zone has also dramatically expanded to include even bad-taste apps that let you jiggle virtual boobs.

I’m starting to think about new iPhone projects, where I can have a high degree of confidence that the app will be accepted. Don’t have anything to announce, but the gears are turning again.

Zi6 Sample: Rangers vs. AC Milan

I was invited to the Rangers vs. AC Milan friendly match last night, and took along my Zi6.

(Warning: the audio starts quietly and then suddenly gets loud.)

I processed the video in iMovie ‘09. A couple of points:

The action scenes are cropped in iMovie, since we were sitting fairly far back in the stand. The quality suffers a little but I think it holds up remarkably well, all things considered. The original export looks even better than the Vimeo-processed file. You can download that file at the Vimeo page.

The Zi6 microphone coped extremely well with some very loud input volumes.

One Hundred Meeeelioooonnnn Photos

Rev Dan Catt writes, on the Flickr code blog:

Over the weekend we broke the Hundred Million geotagged photos, actually 100,868,302 at last count, mark. If we remember that we passed the 3 billion photos recently and round the figure down a little that means (does calculations on fingers) that around 3.333% of photos have geo data, or one in every 30 photos that get uploaded.

In the last two and a half years there have been roughly as many geotagged photos as the total photos upload to Flickr in its first two years of existence.

This is starting to get seriously, seriously interesting. Combine that with GPS-enabled devices, and you get a fascinating data set to explore.

Darkslide uses this data in two ways: the smash-hit Near Me feature, which shows you photos near your current location, and Places, which lets you search for a place and see photos tagged in that area.

I can’t wait for the day when GPS is a standard feature of all cameras, but in the meantime, Derrick Story has been writing about the Jobo PhotoGPS hot-shoe GPS unit, which looks quite interesting and a lot more convenient than some other solutions.