Yahoo Woe (Where On Earth, that is) IDs.

Roll up, roll up, roll up, get your WoE IDs here …

http://developer.yahoo.com/geo/

… a jolly nice step in the right direction.

Yahoo have opened up their geo database (which is pretty good btw) which is far more awesome than I’m going to make it sound in this blog post. I already did a little bit back here. But a quick call out to these two functions that’ll try and find you WoE IDs based on string input…

… the first API call gives, 36240 as the value for Stoke on Trent in the UK, my old hometown. With that WoeID I can plug it back into these other API calls to get various useful information back …

We use WoE IDs over at Flickr, again you can read a little more about that near the end my terribly long previous blog posts about twitter, APIs and such like. A quick recap is that we have these WoE related APIs …

The ones that probably compliments the APIs over at developer.yahoo.com/geo are flickr.places.findByLatLon, which will turn a Lat/Long into what Flickr believes is there, more on this in a second.

And flickr.places.resolvePlaceId, if you plug the value of 36240 into the API explorer there you’ll get this xml back …

<rsp stat="ok">
	<location name="Stoke on Trent" woeid="36240" place_id="gEXCB1iaB571gw" place_url="/United+Kingdom/England/Stoke+on+Trent">
		<locality place_id="gEXCB1iaB571gw" woeid="36240">Stoke on Trent</locality>
		<county place_id="B_K1Z7iYA5qfCIiHaw" woeid="12602189">Staffordshire</county>
		<region place_id="pn4MsiGbBZlXeplyXg" woeid="24554868">England</region>
		<country place_id="DevLebebApj4RVbtaQ" woeid="23424975">United Kingdom</country>
	</location>
</rsp>

Where you can see the same woeid, the URL to get to the Places page for Stoke on Trent, and the parent hierarchy flickr uses.

You can also plug the woeid into the flickr.photos.search to get photos back for just that area. A quick example of why that is useful is California, there’s no way you’d want to find photos for California using the bounding box, as it covers a couple of other states …

california.jpg

… but instead you can use the woeID for California (2347563) which deals with all the bends and kinks of CA. You can also throw that ID into the Places URL, like this … http://www.flickr.com/places/2347563, although you’ll probably want to pass that through flickr.places.resolvePlaceId first if you want a pretty URL.

Differences between Yahoo geo API stuff and Flickr

Over at Flickr, we only use specific ‘levels’ of geo information, such as city, region, state, country, while the APIs over at Yahoo will spit out far more levels in-between the ones Flickr uses, as well as deeper down to neighborhood levels, which Flickr doesn’t do (yet).

Just because you get a WoeID back from Yahoo, doesn’t mean we’ll have photos for that specific area, we’ll probably bounce you up to the next largest places we deal with. So our parent hierarchy will have less steps in it that the ones you get back from Yahoo’s geo stuff. It also means our find by lat/long will only go down to town/city level and not precise neighborhood level.

We also don’t do photos or Places pages for Landmark WoeIDs, taking their example of Sydney Opera House which gives you an ID of 28717584 and throwing that at Places will give you no photos (maybe one day), although you can bounce up to Sydney.

Anyway, it goes beyond photos at Flickr, it’s just a really useful way that you, a developer can key off an ID for a place, that someone else, somewhere else can also key off.

If you use the ID 2347563 for California, and they use the ID 2347563 for California. And one, or both of you, publish your information with that ID, then you, they, or other people know that you’re both talking about the same place and match that data up.

Which is nice.

[Update 1: If you're coming here from the CNET news article my talk isn't actually about the location platform, its about Flickr photos and location :)]

[Update 2: For more official stuff about the Location Platform, you should probably see the Yahoo Local & Maps Blog post Abstracting Spatial Relationships with the Yahoo! Internet Location Platform where they use phrases such as "unambiguously", "permanent" and "language-neutral", and sum everything up far better than I]

[Update 3: and if you haven't yet checked it out, I still think its worth reading my blog post "Twitter API updates, FireEagle and new Flickr API fun" as an example of what to do with woe IDs]

Google Map Photos and SuperGeotagged

There’s been a bit of coverage about Google’s new addition to maps.google.com, with VentureBeat probably adding the most opinion about it. Google’s video has it pretty much covered …

… Ok, so just a touch deeper at what I think is going on here.

Google snapped up the rather wonderful Panoramio a while back, a photo site built on the raison d’atra of location. And not knowing the full details I believe that Google/Panarmio have a cycle of someone or something, or a combination of both of those, looking at all the new photos (since the last cycle), and deciding which ones will make it into the next Photo layer on Google Earth. Or rather I suspect they decide which ones won’t, but the principle is the same.

The new crop of photos, and removal of old ones that have either gone from the site, or someone has asked to be removed, are then converted and stored in a Layer for Google Earth. So when a version of Google Earth is released, it comes with the latest “Data Pack” of Panoramio photos. There’s no actual ‘live’ searching of the Panoramio site that I’m aware of.

Now, at the same time as making the layer, Google can also prepare the data for Google Maps, let’s take a quick peek at San Francisco …

Google Maps - San Francisco

… here’s a similar view from SuperGeotagged (using flickr photos, more on that later. I also think they could slightly smaller images at this zoom level) …

SuperGeotagged - San Francisco

This is done by loading in an additional set of transparent tiles over the top of the map, this is tangently covered in my previous blog post Paul Smith’s thoughts about maps, it’s worth scrolling down to read his comment at the bottom.

Here’s an example of a transparent tile from both sites …

Transparent Map Tiles, Side by Side

… Google’s tiles are served from their own servers, and SuperGeotagged supply their own slightly less ridged looking tiles.

SuperGeotagged: You can read more about the SuperGeotagged over here: SuperGeotagged - Every Geotagged Photo. Which as it happens isn’t every geotagged photo, just everyone with the “geotagged” tag, around 1.5 million of them. I have no idea how long it took Sean to render the tiles, but he needed to grab the thumbnail for each photo, so I suspect a long time.

Google on the other hand could possible render the tiles on the fly, if you compare the two images below, the first from Google and the second from SuperGeotagged, it looks like Googles coverage is quite sparse, which is isn’t. Actually they seem to have just deciding to render only a certain number (sorted somehow) of photos at a time, for if they are using the same photos in maps.google.com as they are in Google Earth, they should have over 3x as many photos represented here than you see on the SuperGeotagged map.

The difference you see below is fairly dramatic …

Google Maps, baked photo tiles

SuperGeotagged, baked photo tiles

… so either Googles tiles are being rendered on the fly (and cached) and only the ‘top’ 200 or so photos are being used to keep thing quick. Or they are all pre-baked, in which case I don’t understand why they don’t go for a more even distribution at the global level. There’s seems to be no reason for google to render the tiles on the fly as no dynamic search is actually being done.

Which brings us to actually how useful is it to display as many photos as possible on a map?

Well with pre-baked tiles, it’s not actually possible to perform searches for images and view them in this style. I can’t just say to either Google or SuperGeotagged, show me photos tagged ‘beach’. SuperGeotagged would have to ask flickr for the photos via API searches and then render the tiles. Google have it slightly easier in that they can have the data on hand, so throwing enough machines at it makes it possible, maybe, sorta.

Each of these sites could pick a theme if they wanted too, “Windmills around the world” and then Google could show them on Global Windmill Day, or something, but then that’s editorial controls, which for the record is just fine with me.

So what are we left with? Well the photos are good for showing us the distribution of where people take photographs (generally where people sell cameras as it turns out), and I believe SuperGeotagged does a better and more artistic job of this. It’s really showing you, “yeah, there’s piles of photos here”.

SuperGeotagged - Geotagging the 280

Google on the other hand, leaves me slightly cold. When you first “Explore the area”, you basically get 8 photos and 2 videos placed live onto the map (the same way that Panoramio do -video), which is great, but at the same time I can’t actually search for stuff, like “coffee”. Or rather as soon as I do, I lose the photos. It being Google, I feel that I aught to be able to search for photos (and the resulting photos to be from the internet as a whole, not just Panoramio, but that’s a whole different kettle of fish).

Then when I click to view “More photos” I get thrown into pre-baked tiles mode, which is great for getting a feel for a place (which you can do with straight image search), which looks like it’s trying to be useful (in a way that SuperGeotagged doesn’t pretend to be), but isn’t so much. I can’t actually click on all those photos of the Golden Gate Bridge because they are all on-top of each other.

Anyway, this is starting to sound like a rant, so I’ll end with this.

I know the people at Google are smart, so I expect something a little smarter. Just because you can throw thousands of photos on a map, doesn’t mean it’s a good idea, no matter how pretty it looks. Unless you’re offering it up as a pretty-thing (like SuperGeotagged), not a search-thing (like what Google is known for).

(and I’m not saying this because SuperGeotagged are using Flickr Photos, in theory with Panoramio’s API they could use ‘Googles’ images, and I’d still prefer the way SuperGeotagged have done it than Google, because of how it’s presented).

Oh and yes I know, its easy to lay out criticism like this, we (flickr) are far from perfect ourselves, with our own collection of problems, but I’m actually in a position to do something about these here. With Google I can just poke, prod and nag, they can always poke, prod and nag back at me next week at Where 2.0 :)

Geotagging Goes Mainstream …

… you know officially!

Before today we were like totally niche, but now that Microsoft has declared “Geotagging Goes Mainstream” I guess we’re out in the spotlight now. Next thing you know we’ll have major newspapers writing articles about geotagging and everything!

All kidding aside this looks like a pretty good tool. I say looks as I’ve not had a chance to play with it yet, being at home with just a Mac to hand, I’ll give it a whirl tomorrow and see how it goes. One of the nice touches that we’ve all talked about, but I’m not sure if anyone really got round to doing well is having a time-offset-slider, I think you can see it in the screenshot below …

PPTtrackroute.jpg
screenshot courtesy of microsoft

… where you upload your tracklog and it matches the time with the timestamps on your photos. The theory being that if you’ve forgotten to adjust your camera for daylight saving, you can shift all your photos along the track until they’re right.

They also suggest that when you drop a photo on the map, be it by hand or automatically, it’ll also add location tags to the photo for Street Address, City, State, and Country. I’ve no doubt that they can get the city right most of the time, but I’m interested how they reverse geocode the lat/long to a street address on a global level.

Which reminds be I should get round to writing that “why reverse geocoding is hard” post at some point, maybe Microsoft have solved it, who knows? Like I said, I’ll take a look tomorrow with some of our known problem areas :)

Anyway, from my point of view, this is a great thing. If a user uses it to catalog their photos with tags and so on in the “metadata” and does the whole geotagging thing too and then uploads those to flickr (or to Microsoft’s photo sharing site thing, do they have one of those yet?), so that it extracts the tags and geolocation stuff, then that makes me happy. Anyway the more people that find geotagging easy, the better it is for everyone else.

And the more photos they can pull back out of Flickr for Photosynth ;-)

Twitter API updates, FireEagle and new Flickr API fun …

You know, just to cover all bases :)

[update: Aaron talks about a similar Flickr/Dopplr/FireEagle dance over here]

Tonight twitter released their next batch of API improvements, of course the one that caught my eye was …

“[NEW] /account/update_location.[xml|json] - sets the location for the
authenticated user to the string passed in a “location” parameter.
Nothing fancy, no geocoding or normalization. Just putting this out
there so developers can start playing with how geolocation might fit
into their Twitter applications.”

… which is nice as it’s just thrown in there as a ‘what if’ type of thing. There’s no direct reason for twitter to have location stuff, (well no more than Flickr I guess) but everyone knows that everyone wants it.

As it says there’s no geocoding etc. it’s basically a free text field to put stuff in that everyone’s agreed is for location. Say you wanted to tell other developers that you were in San Francisco when you twitted a tweet, you could use a URL like this … (don’t click this link unless you want to update your location to San Francisco!)

http://www.twitter.com/account/update_location.xml?location=san+francisco

(This will pop-up a username/password box, that’s twitter asking, not this website btw, you don’t have to enter your username and password, all that’s going to happen if you do though is it’ll set your location to ’san francisco’)

… the return looks something like this …

twitters new update_location method

It’d be great if you didn’t have to update twitter yourself and there was something else out there that could do it for us.

Fire Eagle seems like a great place to start. For some absurd reason I can’t fathom, too busy probably, I’ve not written about Fire Eagle. I think it’s the most wonderful thing in the world, but that aside, it is, in short, a location broker. You get things to put your location in, and allow other things to get location out.

I tend to have Zone Tag running in the background on my N95, set to ping updates to fire eagle every few minutes (or until the battery dies). I hear that navison is also very good. Frankly, I want anything that knows where I am to be able to tell Fire Eagle, if it can’t do that, then what’s the point? I fully hope to have about three hardware devices and a couple of web applications (like Dopplr) sending updates to fire eagle by the end of the year.

Ok, so assuming Fire Eagle knows where I am, I can then authorize another application to query my location (to a level of granularity I specify). When that application asks where I am, it’ll get a response back something like this (example from here) …

<rsp stat="ok">
	<user token="abcdefghijkl">
		<location-hierarchy>
			<location best-guess="true">
				<id>114031</id>
				<georss:point>37.7812461853 -122.3957595825</georss:point>
				<level>0</level>
				<level-name>exact</level-name>
				<located-at>2008-03-03T10:58:55-08:00</located-at>
				<name>500 3rd St, San Francisco, CA</name>
			</location>
			<location best-guess="false">
				<id>114041</id>
				<georss:box>
				37.7494697571 -122.40650177 37.7862281799 -122.3790893555
				</georss:box>
				<level>1</level>
				<level-name>postal</level-name>
				<located-at>2008-03-03T10:58:55-08:00</located-at>
				<name>San Francisco, CA 94107</name>
				<place-id>8Xq01wWYA5u_OEMhyQ</place-id>
				<woeid>12797158</woeid>
			</location>
			<location best-guess="false">
				<id>114051</id>
				<georss:box>
				37.7037811279 -122.5154571533 37.8545417786 -122.32472229
				</georss:box>
				<level>3</level>
				<level-name>city</level-name>
				<located-at>2008-03-03T10:58:55-08:00</located-at>
				<name>San Francisco, CA</name>
				<place-id>kH8dLOubBZRvX_YZ</place-id>
				<woeid>2487956</woeid>
			</location>
			<location best-guess="false">
				<id>114061</id>
				<georss:box>
				32.5342788696 -124.4150238037 42.0093803406 -114.1308135986
				</georss:box>
				<level>5</level>
				<level-name>state</level-name>
				<located-at>2008-03-03T10:58:55-08:00</located-at>
				<name>California</name>
				<place-id>SVrAMtCbAphCLAtP</place-id>
				<woeid>2347563</woeid>
			</location>
			<location best-guess="false">
				<id>114071</id>
				<georss:box>
				18.9108390808 -167.2764129639 72.8960571289 -66.6879425049
				</georss:box>
				<level>6</level>
				<level-name>country</level-name>
				<located-at>2008-03-03T10:58:55-08:00</located-at>
				<name>United States</name>
				<place-id>4KO02SibApitvSBieQ</place-id>
				<woeid>23424977</woeid>
			</location>
		</location-hierarchy>
	</user>
</rsp>

For the moment, let’s say I’m sharing city level data with twitter, we’d be looking at this snippet here …

<location best-guess="false">
	<id>114051</id>
	<georss:box>
	37.7037811279 -122.5154571533 37.8545417786 -122.32472229
	</georss:box>
	<level>3</level>
	<level-name>city</level-name>
	<located-at>2008-03-03T10:58:55-08:00</located-at>
	<name>San Francisco, CA</name>
	<place-id>kH8dLOubBZRvX_YZ</place-id>
	<woeid>2487956</woeid>
</location>

… there are two useful location IDs you can use, place-id and woeid, woeid is probably the best one to use going into the future. In short it’s a unique identifier that Fire Eagle uses to pin a place to a record in the database (there are several San Franciscos around and they’re really hard to disambiguate without at some point going to a unique ID).

The main point is, if everyone can agree that 2487956 is San Francisco, CA, US, and I twittered from 2487956 five minutes ago and you twittered from 2487956 two minutes ago, then we are both …

  1. In the same place
  2. In San Francisco, CA, US

So how would this work? Well probably something like this …

You have a twitter app running on your PC, Mac, iphone or whatever, and your iphone just so happens to be running navizon which is updating your exact lat/long position to fire eagle. You then have a twitter application that you’ve authorized with fire eagle, allowing it to see which city you’re in. When you send a twitter the application first sends a request to fire eagle for your location. It extracts the woeid (or place-id) and before it sends the tweet to twitter calls twitter’s update location method …

http://www.twitter.com/account/update_location.xml?location=2487956

… then sends the tweet. Now other applications that display your tweets can also grab your location information from twitter at the same time and maybe put your tweet in a little more context. I know that I often use twitter to home in on people for meeting up.

Now there are a few problems with this …

1) Each tweet isn’t location-stamped, it’s only your own location that’s updated, looking back you wouldn’t get a historical view of where the tweets happened, so if you twittered something in Paris last week, and then went to San Francisco, the only location an application would have access to would be the San Francisco one. Unless you had something tracking that and recording it for you, i.e. something like twitterrific could save the tweets and locations at the time it received them locally, and allow the person viewing a history of your tweets to see them on a map, or whatever. I don’t think twitterrific currently does keep a history of tweets, but if it did, etc. etc. Hopefully in the future with another nudging we’ll be able to location-stamp the tweets with something like <located_at>.

2) Saving 2487956 into the location isn’t very user friendly, it doesn’t really mean anything to someone viewing your page on twitter, So far I’ve just been looking at this from a code point of view. A solution could be to store something like “san francisco woeid:2487956″ or “san francisco place_id:kH8dLOubBZRvX_YZ”. Then whatever is looking for your location can parse the location based on those patterns. The human just deals with “parsing” the san francisco bit of it.

saving woeids in the location field

3) Fire Eagle currently offers no way (as far as I know) to find out where 2487956 actually is. What’d be really handy is something where you can say, hey where on earth is 2487956.

But anyway, the short is that the new update_location method in the twitter API opens up some possibly fun options. Fire Eagle is a good way to handle getting the location, because ZoneTag that knows and cares nothing for Twitter, is constantly updating fire eagle anyway. And it’s relatively trivial to have something else asking for your location back from fire eagle and pushing that too twitter. Before fire eagle you’d have to beg the people behind something like ZoneTag to also update your twitter location, and then the next thing and the next. Now all they (or any other location type thing) needs to do is send updates to the one true fire eagle service and everyone else can tap into that.

I was going to write a proof of concept myself, probably involving something cheeky like having Google Reader point to a php page that was pretending to be an RSS feed (poor persons cron job), but actually did the request to fire eagle and update to twitter. But I decided just to write a blog post about it instead.

But wait! There’s more.

Now seems to be a great time to mention a handful of new Flickr API services. We have four new Places API methods (and have had for a while now), that you can find on the services/api page, under the Places heading.

flickr.places.resolvePlaceId most closely follows the above discussion. Even though it’s called resolvePlaceId, it’ll also handle woeids. So calling it with place_id=2487956 will get flickr to tell you where it thinks that location is. Plugging the values into the API explorer will get you something like this …

Flickr Api Explorer - flickr.places.resolvePlaceId

So if you parsed woeid:123456 from someone’s twitter profile xml throwing it at resolvePlaceId could get you back a friendly name.

It should be noted that Flickr has no intention of becoming an all purpose general geocoder and as such all these places api methods only work down to a city level. This api call is used mainly for when you want to be able to direct a user to the Places page for a given photos. For example using the flickr API explorer for flickr.photos.getInfo enter the photo ID “2434908810″ and select “Do not sign call?”, in the XML that you get back there’s a section for location …

<location latitude="37.783142" longitude="-122.40411" accuracy="16" place_id="kH8dLOubBZRvX_YZ" woeid="2487956">
	<locality place_id="kH8dLOubBZRvX_YZ" woeid="2487956">San Francisco</locality>
	<county place_id="hCca8XSYA5nn0X1Sfw" woeid="12587707">San Francisco</county>
	<region place_id="SVrAMtCbAphCLAtP" woeid="2347563">California</region>
	<country place_id="4KO02SibApitvSBieQ" woeid="23424977">United States</country>
</location>

You can now take the woeid for this picture, plug it into flickr.places.resolvePlaceId and you have the URL for the places page.

As an aside, once you know the woeid for a photo, you can call flickr.photos.search and pass in the woeid as a parameter to find other photos in the same location, which is also a new thing. Handy for finding photos in California (2347563) where a bounding box isn’t going to help you much.

flickr.places.resolvePlaceURL takes you back the other way. If you know the Places page URL, you can use this to get back the woeid and place_id. Or you can try hand crafting them, such as “/us/ca/sf”.

flickr.places.findByLatLon for flickr type stuff is used to get the woeid at a certain point (city level and above), once you have the woeid you then go on to use flickr.photos.search to find photos in that area.

In the context of updating your twitter location above without fire eagle, then you could pass your current lat/long to this method, get the woeid back out and update your location with that. Another application could then read your woeid back out of your location and using flickr.places.resolvePlaceId convert it back to a human readable format.

Round trip example …

Plug, lat/long 37.783,-122.404 into flickr.places.findByLatLon, gives you …

<rsp stat="ok">
	<places latitude="37.783" longitude="-122.404" accuracy="16" total="1">
		<place place_id="kH8dLOubBZRvX_YZ" woeid="2487956" place_url="/United+States/California/San+Francisco" place_type="locality"/>
	</places>
</rsp>

So now you have the woeid. You can either parse the place_url to get the human readable format, or push it back into flickr.places.resolvePlaceURL, doing that for “/United+States/California/San+Francisco” gives you back …

<rsp stat="ok">
	<location name="San Francisco" woeid="2487956" place_id="kH8dLOubBZRvX_YZ" place_url="/United+States/California/San+Francisco">
		<locality place_id="kH8dLOubBZRvX_YZ" woeid="2487956">San Francisco</locality>
		<county place_id="hCca8XSYA5nn0X1Sfw" woeid="12587707">San Francisco</county>
		<region place_id="SVrAMtCbAphCLAtP" woeid="2347563">California</region>
		<country place_id="4KO02SibApitvSBieQ" woeid="23424977">United States</country>
	</location>
</rsp>

Which gives you a little bit more to work with. Then you can throw that at twitter’s API …

http://www.twitter.com/account/update_location.xml?location=san+francisco+woeid:2487956

Other people can grab your location using the twitter API, parse out the woeid and throw it back into flickr.places.resolvePlaceId to get the human readable form. Of course as it also gives you the state, and country, you can find out that even if you’re not twittering in the same city, you may be in the same state or country. You could also probably work out which timezone they are in, to put some twitters into context.

Oh and you’ll probably want to watch your flickr API rate limit too (hint: local caching) before your key gets turned off.

So there we have it, a quick poke at one of twitters new API methods, a look at how fire eagle could easily be used to keep twitter updated with your current location, without having to go to all the trouble of updating twitter itself. And a quick run through of some of Flickr’s new API methods.

Hopefully if you add into the fray Upcoming.org that uses place_ids such as this http://upcoming.yahoo.com/place/kH8dLOubBZRvX_YZ (spotting that’s the place_id for San Francisco from all the examples above) you should be able to see how fire eagle can be really great for pulling all these things together.

If you use it to keep your twitter location updated, you can start to pull in events from upcoming that happened at that time, see if your contacts/friends were also at the same events, check out flickr photos for that location from your contacts for the timespan of the event. Or go the other way round and have it so that posting a flickr photo to an upcoming.org event, sets your location in fire eagle for twitter to poll for your location, and so on. And if other websites also use the same woeid for news events, and so on and so on, you can see how all these things can be pulled together and support one another.

Obviously its all a little geeky at the moment, but the idea is to make it all nicely seamful as someone might say.

Making Sense of the World, Web2.0expo slides are online.

Maybe a little tricky to follow what’s going on without me wildly waving my arms around but I promised to put the slides up online (I used keynote converted to pdf, so you’re missing my notes, which were mostly swearing anyway) …

… to start with you have to understand 2 things:

  1. The talk was absolutely fascinating
  2. The joke about Pirates was hilarious

Beyond that, until I write it up properly by adding in the bits I realized I missed out or wasn’t clear enough about (thank you to the people who asked great questions at the end) these are the main take away points.

  • If you’re hosting on a single box, you probably don’t need spatial indexing, by the time you need it you’ll have moved into a horizontal server setup.
  • When you’ve moved onto a horizontal server setup, you’ll need some kind of indexing engine and you probably don’t need spatial indexing until you hit around 50 million documents, it’s just not really worth it, unless you’re doing something really complex.
  • Hilbert Space Filling Curve is used for spatial searches. Until you start hitting limitations you can happily index on latitude and longitude. Using Hilbert’s you can collapse those two indexes down to one, either stopping your queries from taking too long, or freeing up an index for something else, like time, etc.
  • Morton’s Curve is used for clustering, not spatial searching.
  • Neither the Hilbert or Morton curve are optimal solutions, but they are good and easy enough.
  • Morton’s Curve doesn’t magically do the clustering for you, you still need to build it into your clustering methods, there’s a billion other ways to do clustering. Morton has a certain elegance to it though
  • Reverse geo-coding is a lot harder than you think it is

I’ll probably write up the Reverse geo-coding part first, as that’s the one that’s possibly more interesting than space filling curves, but who knows :)

done - geo talk at web2.0 expo

And now I’m in a burger joint getting food and tapping this out on a phone.

I’ll get slides and supporting material up over next week once I’ve removed all the swearing from the notes :)

[worklog] Little Wins - “Link to this” back on the map.

It’s been a bit of a stop-and-start past few weeks. You knows, where it’s not possible to quite get into your groove. I’ve sort of been circling around the geo-stuff, not quite able to get my teeth into it. So in the meantime I’ve been picking off general Flickr site bugs from the queue and various things as they popped up, trying to get myself a clean plate from which I can mentally move forwards.

Here’s “Link to this” …

Linky goodness.

… yes it’s a small feature, one that got lost in the new maps shuffle and it’s also nice to have it back. There’s a handful of these little features for the maps and Places, that didn’t quite make it into the last release, or, based on feedback seem like good additions.

So this is kinda getting back into the swing of things. Hopefully there should be a few more little wins soon, because, well, that makes me happy.

Geotagging Video (on Flickr)

So, yay, Flickr released Video today.

One of the things I really like about this is that they get geotagged in just the same way, you drag them onto the map, or you use external tools that call the Flickr API to set location …

flickr.photos.geo.setLocation

… just as you did before. The thing it doesn’t do is read location data from EXIF headers, because, ummm, video doesn’t really have they yet. We’re only just seeing the results from the demand for embedded GPS in cameras (mainly in mobile phones, but whatever), so hey, why not start it for Video too.

And, you get to view videos mixed in on the map just as you do photos (some of my videos in SF) …

Video on the Map.jpg

… Obviously there’s the question of how you geotag a video that’s travelled over a distance, but for my favorite use for geotagged videos; a simple shot of a location, with the sounds of the environment (birds, trains, passers-by) in the background or someone narrating the view, essentially a photo with a soundtrack, it’s great.

You can of course pull geotagged videos out via the API the same way you do with photos …

flickr.photos.search&bbox=-180,-90,180,90&media=videos&extras=media,geo&user_id=35468159852@N01&api_key=your_api_key

… but with the new media=videos attribute, extras=media,geo will return the media type (photo/video) in the records you get back (the geo part gives you the lat/long), pretty sure the online docs will get updated any time now ;)

So now we have around 45 Million geotagged photos, lets see what we can do with video to build up a resource of API searchable geotagged video, with the ability to do the usual search parameters; tags, bounding boxes, creative commons, etc.

Paul Smith’s thoughts about maps.

List Apart have a good article up for people looking to move beyond just-throwing-a-Google-Map-onto-a-webpage. The general gist is summed up well here …

“The result is Google Maps fatigue. We’ve all experienced it. It manifests not only when we yawn at YAGMM (Yet Another Google Maps Mashup), because there are high-quality web apps deploying the Google Maps API seamlessly and with great success. Despite this, and despite the fact that Google itself continues to refine and improve the base application, the fatigue remains.”

(Obviously this is not just a Google thing and applies to any insta-embed-a-map-a-tron. You may think that “insta-embed-a-map-a-tron” is a made up word, but it’s totally not as it’s in Wikipedia so must be true. Well true until they delete it anyway.)

Paul’s talking mainly about the aesthetic considerations … why when you’ve put so much time and effort on making your site look good and unique do you allow a great big chunk of Google/Yahoo/Microsoft branded real estate onto it … but by also talking about rolling your own Map Server it means that hopefully we’ll start to see maps having greater integration with the site they are sitting in.

I know that sounds odd, because you can already do a lot with the various mapping APIs around, but they still tend to dictate how the mapping-app is developed because of the API methods allowed. I sometimes find myself digging around in private methods of various APIs to get the map to do just the very thing I want. But, if I could roll my own, and really knew what I wanted to do before I started (always a good thing) then building it in bottom-up would allow me to do ‘new-things’.

However, I doubt we’ll see too many people taking up that challenge particularly soon.

However (because I’m in a howevering mood) two of the ingredients are certainly worth poking around with even if you have no plans to roll your own mappy thing.

1: TileCache …

A List Apart_ Articles_ Take Control of Your Maps.jpg

… from where you serve tiles. Pointing to some alternative tiles is particularly easy, I used this bit of code the other day (Yahoo!) …

YMapConfig.tileReg=['http://xxx.xxx.maps.yimg.com/png?t=m&','http://xxx.xxx.maps.yimg.com/png?t=m&'];

… to test some new tiles that had just been rendered in some far distant land. They looked lovely and lined up just right btw.

It’s unlikely you’re going to want to render out all tiles, but using off the shelf mapping APIs it’s really easy to limit the maps zoom range and bounding box limit. So if you wanted your site to be about a specific area, with it’s own look and feel, then you can still do that relatively painlessly. Paul points to TileCache in the article as a starting point for those looking to serve their own tiles (including from S3, etc.)

Then, you have to make your tiles …

2: Making Tiles.

Well, you can draw them by hand (which isn’t as dumb as it sounds btw). Or a good source, if they have it covered is my old favorite Open Street Map (OSM), which contrary to my previous post we do need (and if they don’t have it covered, then you should cover it).

The great thing about using OSM is that when it comes to rendering the tiles you can pick your own colour scheme, which instantly separates it from the “insta-embed-a-map-a-tron” (is a real word) maps.

Fact 1: Rending your own tiles from OSM is pretty hardcore … here are links …

OsmaRender
A rule-based rendering tool for generating SVG images of OSM data.
Mapnik
An Open Source map renderer which we use to render the main Slippy Map layer for OSM.
Kosmos
A lightweight OSM map rendering platform developed by Igor Brejc
Making the nestoria tiles.
A page detailing how OpenStreetMap was integrated into Nestoria, the first commercial use of OSM maps

Fact 2: It’ll probably be quite hard and throw up a bunch of wtf? questions. If you solve them, or find simpler ways of doing it, then it’ll help lots of other people. Which is a good thing.

Anyway, this was just supposed to be pointing to the article in a [This Is Good] way, but of course I have DVD-Extras syndrome, so you have my above ramblings.

Andrew said it better, which is where this is [via].

libkml 0.1 ALHPA - Google Release Thing to make making other thing easier …

… or at least reduce the risk of errors while creating things.

Ah, KML, otherwise known as a Google Earth (Network) Layer thing, previously Keyhole Markup Language thing and now edging towards a useful format for transporting geodata around thing as the OGC (Open Geospatial Consortium) nudge KML closer to standards and out of the Evil propriety clutches of Google.

Propriety being Evil here, not Google of course.

Anyway, freshly released is Google’s libkml, a handy library that you can use to create KML (and then zip up for KMZ). You can read far more about it over at High Earth Orbit. Like Andrew …

“Unfortunately since my laptop hard drive died last week, I don’t have a development machine to build and play with this yet. But I expect to use this library in a number of projects.”

… I’ve not had a chance to play with it yet. Not so much because my laptop is broken, because it isn’t (and boy I wish it was so I could get the IT department to get me a new one), but because as anyone who knows me knows, I moan bitterly about having to type anything like build, make or in this case scons into a console. As much as I love Macs I’m no doubt still scarred from when I tried to compile PHP with some graphics libs in, anyway I digress.

Why am I happy about this, well, mainly because it’s another indicator of the progress towards KML being a standard and less Google’s thing. Which means that I don’t have to make quite so many arguments for why we should support it, being a Yahoo house and all (not that I’ve ever had to have made those arguments, but you know). Having a library means I can say “But look, we have a library”. Simple I know but never underestimate the power of a library.

Anyway, I’m rambling, basically because High Earth Orbit has it all covered in detail, so stop reading this and read that instead.

Next entries »