10 October 2012
Many months ago, we were approached about helping start up a LocalWiki project in Antarctica.
The project, named Open Antarctica, would aim to initially document a region of roughly 2 miles surrounding the Palmer Station United States base on the Antarctic Peninsula.
Our first question was: wait, there's Internet in Antarctica? And yes, there is Internet access on the Antarctic bases — access that's provided over an awesome looking, though incredibly slow, satellite uplink. So slow, in fact, that we briefly thought about sending our contact down to Antarctica with a plug-in computer running a little LocalWiki server — but ultimately decided it was too much work for little gain.
The next thing we started thinking about was mapping. Maps are a core feature of LocalWiki, so having a good map of the area, particularly the area near the research station, was really important. And ideally we'd have some aerial imagery and not just a vector map. "Let's take a look at some maps of the area!" we thought.
Given that we use OpenStreetMap data in our default base layer, let's check out what our OpenStreetMap-based tiles look like around the island containing the research station:
As you can see, the OpenStreetMap map doesn't really show any of the detail here — it's missing the whole peninsula the station is on! The story's the same with most freely-licensed satellite imagery:
Some commercial satellite imagery, like Google's, is better, but it's still not at the level of detail we wanted — and it's not freely licensed. There's got to be some good aerial imagery of this region out there, right? Maybe if I we found some good imagery we could stitch it together into a map?
Then we found this amazing NASA Photo of the Day:
OH MY GOD! That's just incredible detail and quality! Hope!
Making the map
Searching through the USGS Earth Explorer didn't turn up anything near the quality of this NASA photo of the day image. Hunting around different NASA websites didn't seem to give many clues, either. The best clue was the photo caption from the Photo of the Day, which read:
[...] greeting to scientists and flight crew aboard NASA's DC-8 flying science laboratory as it flew over the station during Operation Ice Bridge.
After even more digging around, I stumbled upon the FTP site where NASA was hosting Operation Ice Bridge images. Presumably, the image from the NASA Photo of the Day would be in there — and hopefully a bunch of great shots of the surrounding area. The only problem was that the FTP site had many, many directories, each with thousands of files:
I experimented with downloading a few random images and plotting them in QGIS, but it quickly became clear that downloading all of the hundreds of thousands of images and plotting them using this way just wasn't going to work — there were too many images, we didn't have the many, many terabytes of required space, and the GeoTIFFs quickly overloaded QGIS. Alex Mandel recommended trying to find a pattern in the filenames — but after working on the problem for a bit we didn't seem to get anywhere.
So we ended up writing a little FTP client to grab just the few Kb of each file, which ended up being enough to get the location metadata out of the GeoTIFFs. With that, we could easily import the location of every image:
...and then start plotting them on a map, revealing airplane's flight path:
Getting there... after plotting a few hundred thousand other data points:
Sweet! It looks like there's quite a few images around the station! So there's some hope we can piece together a great aerial map of the region.
Overview / vector map
So, there's hope we'll pull off an aerial map of the region. But it's going to be pretty spotty — the NASA Ice Bridge project is set up to just snap photos of the coastlines, basically. We'll have to lay the aerial imagery on top of something. And because the project may later want to cover other regions in Antarctica, we'd like to have a good base map of the whole continent.
But the problem is that standard maps are designed to make most of earth look right when flattened out, always at the expense of Antarctica. Which usually makes a lot of sense.
Paths around the South Pole would look really wonky with this standard projection. Fortunately, there's a better projection for Antartica. After much fiddling with Geoserver, we figured out how to get it to correctly reproject vector data in this esoteric projection.
So, we can reproject vector data into this fancy Antarctic projection, but where can we get decent vector data of Antarctica? As explained earlier, the OpenStreetMap data isn't very detailed. Thankfully, the fantastic Antarctica Digital Database comes to the rescue! They have highly-detailed coastline data of the entire continent that's freely usable for non-commercial purposes.
Stitching together aerial imagery
So, we've got all these images from the region around the station, but how do we stitch them together into a map we can serve and suck into OpenLayers?
Figuring out how to serve individual GeoTIFF files was pretty easy. The problem was figuring out how to stitch all of these images together. We tried out several of the Geoserver-specific approaches to image stitching, but none of them worked. The Image Pyramid plugin seemed to crash when using this Antarctic projection — and the same sort of crash happened when trying to import a VRT file of all the GeoTIFFs and when using the Mosaic feature of Geoserver.
Maybe we can just stitch the GeoTIFFs together ourselves, somehow? First, we tried using gdalwarp to do the stitching:
...which worked, but there are weird black lines in the image, what are those? Turns out the images, while TIFFs, were compressed internally using JPEG compression and we need to run the nearblack utility on them before stitching.
So, we threw together a script to clean up the images and continually stitch them together using gdalwarp. All done? Not so fast. The images, when uncompressed, balloon to around 2GB or so each — EC2 to the rescue! We spun up a server with an insane amount of disk and ran the stitch script for several days, taking a look at the output and tweaking it throughout.
This left us with one stitched-together, compressed GeoTIFF that weighs in at slightly less than 2GB. After building overviews for various zoom levels into the GeoTIFF, it was relatively easy to load it into GeoServer as a raster storage source and then combine it with the vector data layer. The aerial and vector layers didn't quite line up, but after fiddling a bit with the projection we got them be as aligned as could be expected.
After much tweaking and dealing with strange errors — almost all of which related to map projection issues — we finally put together a beautiful map of Antarctica in a glorious stereographic projection with stunning aerial imagery of the region laid on top of it:
Some beautiful pages
You should explore the project! Here's some beautiful pages to take a look at:
31 August 2012
We're excited to announce that the first version of the LocalWiki API has just been released! If you're running a LocalWiki instance, please upgrade to get the new functionality.
What's this mean?
In June, folks in Raleigh held their annual CityCamp event. CityCamp is a sort of "civic hackathon" for Raleigh. During one part of the event people broke up into teams and came up with projects that used technology to help solve local, civic needs.
What did almost every project pitched at CityCamp have in common? "Almost every final CityCamp idea had incorporated a stream of content from Triangle Wiki," said CityCamp and TriangleWiki organizer Reid Serozi.
The LocalWiki API makes it really easy for people to build applications and systems that push and pull information from a LocalWiki. In fact, the API has already been integrated into a few applications:
Another group in the Raleigh-Durham area, Wanderful, is developing a mobile application that teaches residents about their local history as they wander through town. They're using the LocalWiki API to pull pages and maps from the TriangleWiki.
Ultimately, we hope that LocalWiki can be thought of as an API for the city itself. A bridge between local data and local knowledge. Between the quantitative and the qualitative aspects of community life.
Using the API
We did a lot of work to integrate advanced geospatial support into the API, extending the underlying API library we were using — and now everyone using it can effortlessly create an awesome geospatially-aware API.
This is just the first version of the API and there's a lot more we want to do! As we add more structured data to LocalWiki the API will get more and more useful. And we hope to simplify and streamline the API as we see real-world usage. Want to help? Share your examples for interacting with the API from a variety of environments — jump in on the page on dev.localwiki.org or add examples / polish to the administrative documentation.
If you're using the Ubuntu package(recommended for most users), then upgrading takes just these two commands:
sudo apt-get update sudo apt-get upgrade
If you have a developer install of LocalWiki, read about upgrading on the developer site.
CityCamp photo courtesty CityCamp Raleigh
By Mike Ivanov
25 July 2012
We want to let you know about some really fantastic stuff that's been happening in the LocalWiki world over the past month!
The folks spearheading the TallahasseeWiki project held their first two in-person CampWiki workshops. The idea behind the workshops is to introduce community members to the TallahasseeWiki, get them excited, answer questions and start building out the project.
Their meetup even made the front page of the Tallahassee Democrat!
OlyWiki held its first little in-person meetup. Unfortunately, they didn't take any pictures, so here's a photo of Seth Vincent, the project leader, putting up some flyers around town!
A group of really great folks have started laying the groundwork for an OaklandWiki project, and during the Code for Oakland hackathon on Saturday there were around 30 people digging into OaklandWiki. The group was so large that it was broken into two rooms, one for helping with development / tech stuff and the other for content and planning!
Raleigh, North Carolina
The Raleigh City Council has begun to investigate how to, in its official capacity, best work with the TriangleWiki project! Here's a clip from their recent City Council meeting where Councilmember Gaylord asks city staff to come up with a report on how to collaborate with the project:
Tokyo area, Japan
Thanks to the internationalization work done by Pedro Lima and Nuno Maltez in Portugal, there's been an increasing amount of international interest in starting LocalWiki projects. A couple of weeks ago, Shu Higashi gave a demo of a LocalWiki to a group of Open Data activists in Japan. The best part? He demo'ed his Japanese translation of LocalWiki!
Black Rock City (Burning Man)
Some folks are starting up a LocalWiki project for Black Rock City / Burning Man! It's literally just getting started, but it's such a cool idea we wanted to share it with you!
LocalWiki Organizers mailing list
Organizing a new LocalWiki project or wanting to get started? You should join the ultra-new LocalWiki-organizers mailing list! Be sure to send a little introduction to the list after you've joined!
In no particular order, a few other interesting things that've happened over the past month:
- Code for America made a blog post which made the case that "open data platforms and local wikis are foundational elements..of civic life."
- Philip gave a talk on LocalWiki at the big Wikipedia conference in DC last week. Lots of excitement & interest!
- TriangleWiki and LocalWiki got a shoutout in the Columbia Journalism Review: Building a multi-platform media for—and by—the public.
Tallahassee photos courtesty Bob Howard. Oakland photos courtesy Eddie Tejeda.
1 June 2012
LocalWiki is all about community, and we believe that starts with people coming together and having fun. Here are a couple of recent examples of ordinary people hanging out, having fun and building out LocalWiki projects!
San Francisco parklet ride
A couple weeks ago we were slated to have an absolutely beautiful weekend here in SF, and I thought, "you know, it's gonna be a beautiful weekend — it'd be great if we could just bike around, take pictures, enjoy the beautiful day, and toward the end of the day get on our laptops and work on the soon-to-be-launched SF LocalWiki project!"
So I just threw up a Facebook event, invited some friends and came up with a little came up with a little plan: document all the parklets we encountered on our bike ride through the city.
And we did just that! We biked around, had lots of coffee, ate some delicious baked goods, and took pictures of a few parklets we encountered. We randomly encountered one of the parklet's primary designers and even found a parklet that features a totally awesome beanbag seats and pacman!
At the end of the day we'd made a few pages on our LocalWiki, but more importantly, we shared a fun experience together!
Kitchener LocalWiki Workshop
This is Steve:
After starting out with the basics of how local wikis work, the workshop immediately got people out into the streets — participants toured the city, collecting information about their favorite places, insider tips, and popular spots. After running about, everybody reconvened and had a big local wiki writing party at the art center.
Kitchener photos courtesy CAFKA.
23 May 2012
While LocalWiki's roots are in the United States, we've seen increasing interest in starting projects all over the globe. One barrier, however, has been that our interface is entirely in English. At least it was, until today:
Thanks to the hard work of Pedro Lima and Nuno Maltez in Portugal, LocalWiki is now completely internationalized and can be easily translated into any language!
Pedro and Nuno have started a beautiful LocalWiki project for the city of Porto, Portugal: por.to. So far, they've been using their LocalWiki to collect information about the remarkable architecture around the city:
We need lots more translations! If you'd like to help with translations, just email email@example.com.
By Mike Ivanov
29 March 2012
Today we are releasing version 0.3, a major update to the LocalWiki software. We're pretty psyched about this release, and you should definitely upgrade just as soon as you finish reading this! Along with many, many bug fixes and improvements, this version includes two really cool features we're excited to share with you: tags and dashboard.
Get organized with tags
LocalWiki's mission is to share the world's local knowledge, and a huge part of sharing this knowledge is organizing it and making it easy to find. Today's software update makes this easier than ever with tags, a painless new way to organize pages.
Wikis are usually organized by creating links from one page to another and by manually compiling lists of pages that have something in common, like "Vegan restaurants". We love and encourage this kind of careful curation, but keeping these lists up to date is time consuming and boring, and creating them in the first place takes someone's initiative. Tags, on the other hand, are a light-weight and even fun way to group pages together.
Tags are like labels that can be added to a page to make it easier to find. When you click on a tag, you instantly get a list of all pages that have that tag, along with a handy map. And you can even include this list in another page, without having to create and maintain the list by hand.
We think you'll love tags! They are very easy to use, yet they are incredibly powerful. You can tag all of the vegan restaurants with the tag "vegan", or tag all the historic sites with "historic", or even tag pages that need a photo with "needs photo"—the possibilities are endless!
With tags, you have a whole new way to explore your local community. You can read much more about tags in our usage guide.
Visualize your progress
Another major challenge when building a community-driven information resource is measuring progress. How many pages does our wiki have? Are we getting new contributors? How are we doing?
We designed our new dashboard to answer all of these common questions and much more. It shows all of the current stats about your wiki as well as some beautiful graphs of different metrics over time, so you can get an immediate feel for where you are and where you are headed.
The dashboard clearly shows you how your outreach affects contributions, so you can set goals that everyone can work toward. In our latest focus community, the dashboard helped the contributors reach an amazing goal of 1,000 pages by launch day!
You can find a link to the dashboard at the bottom of the "Recent Changes" page on your wiki.
There's so much more we could talk about, but we want you to try the new version for yourself!
If you're using the Ubuntu package(recommended for most users), then upgrading takes just these two commands:
sudo apt-get update sudo apt-get upgrade
If you have a developer install of LocalWiki, read about upgrading on our developer site.
14 March 2012
The Triangle Wiki launches with over 1000 pages — more than three times the amount that the Davis Wiki had at its launch. Interestingly, the Triangle Wiki wasn't spearheaded by college students like the Davis Wiki or the DentonWiki. The core group that's driven the buildout of the Triangle Wiki met at City Camp Raleigh and is really far reaching — consisting of everyone from, yes, college students to City Council members.
At over 1000 pages, it can be hard to get a quick feel for the project's breadth. With that in mind, we put together a little random page tour — check it out!
27 February 2012
Whew! A lot's happened since our last blog update! Our first focus community, DentonWiki, has been doing great and several of our other focus communities are close to launching.
Just two days ago, nearly 50 people came together in Raleigh, NC to join a massive in person content-building sprint to build up the soon-to-be-launched TriangleWiki.org. Folks from all walks of life joined in — two City Council members and Raleigh's Chief Planning Director even came by to help out! Read more about the event on our blog.
And our new software has been rapidly adopted by communities around the world. Since our last email, over 182 independent communities have installed our LocalWiki software!
We thought it'd be great to take some time to look back and reflect on our past year. We've put together this report highlighting some of what we've accomplished and where we're headed:
Check it out and share widely!
Philip & Mike
25 February 2012
This blog post originally appeared on southwestraleigh.com, by Jason Hibbets
Almost 50 people collaborated today at Red Hat headquarters, currently located on Centennial Campus in Raleigh, NC, to participate in Triangle Wiki Day. The event was a soft launch of trianglewiki.org, an effort to document the Triangle region and increase collaboration and knowledge sharing across the area. The wiki uses open source software, local wiki, as a content management platform that includes wiki pages, images, and mapping.
The day started off with a brief presentation [PDF] by Jason Hibbets on how the Triangle Wiki project fits in with the CityCamp Raleigh movement, as well as the larger open government picture and civic innovation week, Code Across America, by Code for America.
Raleigh At-large City Councilor Mary Ann Baldwin keynoted at the event. She spoke briefly on the importance of collaborating on a project like Triangle Wiki. She also mentioned that she was wearing multiple hats: City Councilor, a marketer, and a member of the Innovate Raleigh steering committee. Part of Triangle Wiki Day is to start mapping the assets for the #InnovateRal initiative and to be an authentic part of Raleigh’s open source philosophy and open-minded communities.
At-large City Councilor Russ Stephenson and Raleigh Planning Director, Mitchell Silver, were also in attendance.
Reid Serozi, Triangle Wiki project lead, provided the background on local wiki, showing a video from Philip Neustrom, one of the project co-founders of local wiki and daviswiki.org. Then he walked the attendees through wiki 101. We learned how to register an account, create new pages, and edit existing pages. After that, the edit party began.
Right away, people started creating pages, collaborating with each other, and helping one another with wiki best practices, formatting, mapping, and more. By 10:30am ET, #triwiki was trending on Twitter in Raleigh.
The group made a lot of progress. Fueled by Klausie’s Pizza for lunch and a bunch of snacks and soda provided by organizers, here are the results of Triangle Wiki Day:
- 633 page edits
- 100 maps
- 138 new photos added
The event wrapped up around 2:00 pm and contributors continued to add pages after the in-person collaboration with a goal of 1,000 pages by March 14.
While I haven’t been as-involved with Triangle Wiki as I was with CityCamp Raleigh, the event was very fulfilling for me. I believe in the power of open source and collaboration and Triangle Wiki Day was another example of this success. It’s also building upon the momentum of the Innovate Raleigh Summit held on January 18.
The Triangle Wiki is about creating something anyone with local knowledge can contribute to. It brings together people with different skillsets—ranging from tech-saavy know-how to photography, local history to hackers, and much more. Triangle Wiki is basically community knowledge done the open source way.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
Media coverage (pre-event)
- Creators make final push for Triangle Wiki – News & Observer
- Creators make final push for Triangle Wiki – Public invited to ‘content sprint’ – News & Observer
- Triangle Wiki: Call for Contributors – Cary Citizen
- Triangle Wiki: For those who don’t have time to start a blog, but have something to say – Raleigh DLA
- Something Wiki this way comes, contributors sought – News & Observer