Watching your data “grow up”

Now that we have started our closed beta of Qortex – our splendid and brand new communication platform – I would like to talk about what thinking went into creating it and why we needed to build our own platform rather than just use what was out there.

Typical Scenario

Up until Qortex, we did our online chatting at the Plant via iChat/Google Talk or another group chat network, we sent each other the occasional internal mail, some of the discussions happened via mail, some on an online collaboration site, and documentation sometimes went there, too, or to Google Docs, or worse – someone’s hard drive. Issue tracking for software development was happening on another platform and our source code is hosted on GitHub.

So we were spread out over 5–7 platforms, which in itself isn’t necessarily bad, as long as those “realms” are mostly isolated and used by people who never cross over into each others’ realms. Which wasn’t really the case after a while…

Panta rhei

Heraclitus got it right: “Everything Flows”, nothing stays the same, so we started using a social-network-like platform for internal communication. This made more crossing over between realms possible, because now more often than not programmers and designers would discuss many aspects of development, and it actually made discussions deeper and more insightful.

But that also meant that spreading ourselves over 5 or more platforms made it quite difficult to share anything when these discussions happened. And it turned out that none of the platforms we looked into could offer what we needed.

Your data grows up

We thought about how ideas come into existence and how they are shaped and transformed until they eventually get realised (or tossed into the bin). You start with something very ephemeral, maybe a vague idea or a spark of genius that came to you chatting with someone. Later run this by someone – in real life, via mail or a social network, this time in more permanent form the idea has become stored information, that’s when you (want to) get feedback, wich further helps shaping and evolving the thought, then maybe you start to start writing down a full concept, and the idea becomes more tangible more concrete and fixed.

After many, many edits eventually it might get realized into, say, a product. Then you’ll probably want to start documenting it in a more permanent and refined form, on a platform that is more suited for long-term storage (and editing), something like a wiki.

So here’s how your data might be growing up and mature baby → toddler → youth → adult → senior idea → discussion → draft → revisions → document (knowledge) chat → post/entry → versioned knowledge base document (wiki-like)

Another interesting aspect of these idea-discussion-knowledge transformations is that not only does the permanence change, but also the noise ratio drops with each stage the information becomes more refined, more pure content.

Enter Qortex

This means we wanted to provide solutions for all levels (noise and ephemerality) of communication, which meant concretely: chat, posts/discussions and wiki-like versioned documents.

It also meant that we wanted to use the front-end that felt most natural to either task. For the latter two the web browser was just fine for us, but chat has always happened in our local chat clients. Offering a chat feature only in a webbrowser seemed wrong and too limiting. It’s nice to have it as an option when you’re, say, on an iPad, but for comfortable chatting being able to plug any standalone chat application was crucial.

Adding to whatever Qortex can already do in a browser, we felt the need for supporting the XMPP protocol, so we could chat via any Jabber capable client (like iChat/Messages) and still save all chats into Qortex and share or transform them into a more permanent form later. It was quite a chunk of work to get that working even in the basic form we are doing it right now, but I really feel it was worth doing it, and we can build upon that to support all extra features of a modern chat client in the near future.

Other Considerations

Most of us really love Markdown and we all have been using it quite a lot wherever possible. It turns out that many popular platforms actually support some level of Markdown (Basecamp, GitHub, Lighthouse, etc.), too.

When we implemented the text editor for Qortex, we started of with a Markdown only editor, leaving the styled editor for later. We wanted full extended Markdown support (including extensions like code fencing). For the final product a styled or near WYSIWYG editor was just as important, but for us to begin with Markdown is what we would be using.

For the styled editor, we never wanted umpteen features, just basic text editing and basic formatting, nothing to distract you from the task at hand: writing. And because of this we decided to store all entries and comments as Markdown, not HTML. The styled editor thus could only provide for functionality that Markdown also supports, but our aim to provide an editor that lets you focus on writing already made sure that we didn’t have too fancy formatting options there anyway, so it was a perfect match.

The main reason for going with Markdown is that it’s more reusable, its easier to convert (cleanly) into other formats and Markdown itself never gets as convoluted as HTML created by an editor. I have never seen any styled editor for HTML produce clean code after, say 20 edits.

Markdown is limited in the formatting feature set, but we still wanted to add functionality to make entries smarter than plain Markdown. Yet, we didn’t want to interfere or “pollute” the purity of our Markdown code. Our solution was to make the interpreter that’s rendering our Markdown code a bit smarter. For example if you put a YouTube or Vimeo URL in an entry or comment, Qortex will recognize it and automatically embed the video in the rendered HTML, so you can play it back right there in Qortex. The source Markdown stays untouched – it’s just a URL.