Building Heineken’s “Open Your City” Platform

R/GA London recently launched @wherenext (http://openyourcity.heineken.com), aimed at providing venue recommendations based on social media activity, via a tweet response and web app.  The app shows the five hottest venues and a heat-map of activity in the user’s vicinity.  During the discovery phase, we were surprised to find that nobody was successfully cross-referencing genuinely real-time social activity with venue recommendations, so this became our goal.

A prototype helped us assess the quality and usefulness of the different data sources available to us.  Foursquare was great as a venue directory but pretty useless as a source of live activity.  Twitter would be essential as the messaging engine but as an indicator of where people were socialising, Instagram was stronger – these learnings informed the source weighting we gave the data we ingested for the heat-map.

The two main elements of the app – the heat-map and recommendations – are tightly related.  Every Instagram post or tweet that we ingest for the heat is also assessed for its proximity to Foursquare venues nearby.  The closest venue gets a tag, assuming it’s within a few meters.  Tags determine venue rankings, but only last for a short time – the same goes for the heat.  Every heat interaction and tag is disposed of after two hours, which keeps the recommendations fluid.

Ingesting social data on a global scale presented logistical challenges.  Instagram’s API only allows for location based searches within a 5k radius of a geo-point, so we created a tool for mapping cities with circles.  A background task constantly scans each mapped city to pre-cache Instagram posts.

Only 5% of tweets are geo-tagged, and we could only afford a 10% sample of this, but consuming even a 0.5% sample of Twitter’s global fire-hose (via Datasift) is not straightforward – we were regularly disconnected for “consuming data too slowly”.  In the end, each interaction was thrown on a queue, with all processing and database insertions carried out by multiple instances of a background worker.

@wherenext was built as an Azure cloud service comprising an AngularJS front-end (with Google Maps), an ElasticSearch database (for super-fast location based searches), and a .Net backend integrating with Datasift, Twitter, Instagram, Foursquare and some Heineken APIs.  We also built our own service, “FaceOff”, to remove selfies using the OpenCV facial recognition library – primarily for legal reasons, though it also gave us better imagery.