Using Github Repos And Jekyll As A Data Store

Github repositories are the heart of all of my API research. Each of the 200+ areas of my research lives as an individual repository, and I publish most of my raw research here as JSON, and YAML--then make it viewable, and explorable using JavaScript and HTML. Github + Github Pages + Jeklyll is what makes all of this possible.

I have been working professionally with databases for over 25 years--I am a database guy. From 1997 through 2007 I was heavily dependent on my SQL Server database(s). From 2007 through 2017 I am heavily dependent on my MySQL database(s). I predict from 2017 through 2022 I will be heavily dependent on my JSON and YAML data stores available via Github and my own server infrastructure.

Using Github repositories as a data store will not replace my central database infrastructure, but it will augment it significantly. Much like dynamically publishing HTML documents from databases has dominated my web evolution, the dynamic publishing of JSON and YAML documents is what drives much of my public presence during my API evolution. Github allows me to drive the publishing of this data using Github Pages, while using Git to maintain a snapshot of my data stores at any point in time.

The static nature of my data stores is efficient, in that they load fast, and leverage simple web technology (HTML, JavaScript, CSS) to accomplish its objective, whether that is delivering HTML to humans, or JSON and YAML to other systems / applications. The publish / cache nature of these representations of my data works well for my approach to storytelling. I can keep my research moving fast, keeping pace with the fast-changing landscape, or I can employ them as a snapshot that stays static forever, something I may never update.

I increasingly find people don't grasp how it is that I use Github to run my API Evangelist, and the potential of Jekyll and Github when it comes to managing data, especially when it is in the service of storytelling on the web. It's not an approach I recommend everyone put to work, but as a database person, I think everyone should have Github and Jekyll as a data store in your toolbox