Kin Lane

The US Narrative Today

I have always really liked newspapers. Today, I think newspapers are an endangered species, not because they aren’t relevant anymore, but because they’ve been hunted and disrupted. In an always on Internet world, newspapers have some serious soul searching to do, and coming up against tech giants like Facebook, Twitter, and others can’t be easy. Regardless of the changes in the technology landscape, or more significantly the advertising landscape, I feel like newspapers are still relevant and important, but for many reasons that aren’t always commercially meaningful.

I feel like local newspapers can still be a voice for a community. With corporate acquisition and consolidation of local news outlets, and many just being driven out of business, I wanted to better understand what was happening across the news landscape across the United States. So I set out to aggregate stories from across as many news outlets as I could, with minimal investment, and try and construct narrative of what was being said. So I got to work on what I am calling the US Narrative Today.

I first had this idea back in 2005 or so, when I started a website called Poem of the Times which took the top headline from 25 newspapers, and organized them into what I loosely called a poem. I ran this from 2005 until probably 2010, and then I let it fall into disrepair. While I was looking at the Newseum website the idea came back to me, and I set out harvesting all the PDFs from the Newseum, and attempted to extract the headline for each day. This task is easier said than done, and in the end I just couldn’t reliably extract the full title programmatically across 400+ PDFs each day—-it just wasn’t something I could easily do.

After some consideration, I switched to pulling a list of 2500+ newspapers in the United States from Wikipedia, and then wrote a script to spider their website looking for an RSS feed. I found about 900+ of the 2500 news sites had a valid RSS feed, which seems to produce around 450 consistent headlines for each day. Which I then take and randomly order, keeping each local news source associated with the title, then I publish them as a random narrative for each day. The stories aren’t fully coherent, but they have an interesting rhythm and flow all their own, with different opening paragraphs, and rambling narrative each day—kind of like us.

It took a while to filter out junk, noise, and other irrelevant items, and an occasional “glitch in the matrix” still occurs every once in a while, but overall I’m happy with what the project produces each day, without interference or editing. I could invest in some more ML, vocabulary tooling, and other ways to make it fit together better, but I kind of like it the way it is. I’m more interested in seeing it play out over time, and then looking back a year to read an older one. I think this is when it will become more coherent, looking back through time, then drifting back to the each point in time. I don’t know. It may go nowhere. I’m not entirely sure why I did this project, or where it will go next, it kind of has just taken the drivers seat, and has be evolving itself.

Once I stabilized the narrative for each day, and it was reliably pulling the RSS feeds, and it was successfully publishing the page of stanzas for each day, I got to work on adding some visual elements. I decided to incorporate my other Algorotoscope work into the project, pulling a handful of random photos that I’ve applied ML texture transfer models to. It just pulls a random set for each story of the day, with a number of potential repeats always possible, as I only have about 5K overall photos to choose from in the index. I’m working on training more filters (which cost me $$), organizing my photo library, and finding more potential targets for applying filters. Then the hard work will be all about tagging the photos. Once this is done, I can begin associating images to the narrative better, making them more relevant and meaningful to the narrative unfolding each day. Who knows, maybe some day I’ll be producing photos purely based upon what the narrative is coming out of towns across the US.

As I said, I’m not sure what is next for this project—-it is kind of driving itself. Right now I’m just letting it run, and working to increase the volume of Algorotoscope images available, and possibly making the image selection more contextually relevant. However, I don’t doubt some other idea will reveal itself to me, for refining and tweaking how it pulls news, images, and programmatically delivers its drumbeat each day. I just like seeing it work, scrolling down through the narrative each day, seeing what is happening across the country. Some of it you can tell is the syndication of news across corporate news outlets, but much of it is also the regular day to day flow of average communities, often ignoring the world around them, and focusing on what matters in their world. I like this. Somehow it makes me feel somewhat better about this country in the current news climate which is often times so toxic, political, and unfriendly.

URL: The US Narrative Today