IMPORTANT: Most of the things mentioned in the post are unfinished, some aren’t started. Treat this post a guide for what’s going on.
* * *
Let’s talk about Starship.
Starship is my internal codename for the New Ludum Dare website project. It’s a silly nickname that I got used to. Loosely, it’s my way of saying Ludum Dare is a big complicated internet space ship that transforms, sends out probes, and even splits off in to smaller ships. I guess it’s actually metaphorically a lot more like Voltron, but I was watching Star Trek TNG at the time.
Starship is written in PHP, (currently) runs on Apache, and uses MySQL. And yes, we’re on GitHub.
Migrating to a Starship
Before I go in to details about the new website, I need to talk about the old website, and what we’re going to do with it.
Over the past 7 years of Ludum Dare, we’ve accumulated over 21 GB of image data, and a nearly 4 GB database. There are over 50,000 users, nearly 75,000 posts, nearly 60,000 comments, and over 27,000 games and things that have been submitted to Ludum Dare events. And with Ludum Dare 33 on the horizon, those numbers are going to get even bigger.
That’s a lot of data.
Today, Ludum Dare runs on a dedicated server somewhere near Washington, DC. This machine is entirely ours, has some crazy Intel Xeon CPU and 12 GB of RAM. We’re also using CloudFlare as a reverse proxy and CDN. That means no matter where you are in the world, those 21 GB of images should load nice and quick for you (here’s a map). The whole setup costs a bit over $200 a month to run.
That’s how we do it today.
For the new website, we are going to start with a clean slate, *BUT* we are keeping the old data.
First off, when we finally kick-off the new website, everybody is creating new accounts.
After you create your account, we’re going to let you migrate your old data. I want this to be as simple as possible, automated even. I’ll autodetect if you have old data by comparing your current and original e-mail address. There will also be a way to claim your data if your e-mail address has changed (process TBD).
There will be a script in place, so old URLs like these:
Will redirect to the new URLs like these (NOTE: links are examples. They are not live):
For accounts that haven’t been migrated, they’ll instead be redirected to a read-only version of the original pages on the Legacy Server.
The Legacy Server will be a separate mini-server (mini compared to what Ludum Dare has become) that runs a read-only version of the original website. Posting and Commenting will be disabled on the Legacy Server, and along the top of the page will be notices telling people that we’ve moved.
Traffic should be minimal, so like the old days, I expect we’ll be able to run it for around $10 a month. Pretty cheap.
That said, the Legacy Server is really only needed for the inactive users. Active users will have all their games and posts available on their new accounts, on the new website.
One caveat: Comments will not be migrated over. Instead, we’ll attach a special link to the original post on the legacy website, where you can read the original comments.
So again, we’ll be starting with a clean slate, but everything from the past will be available.
* * *
FYI: For anyone that’s curious about the legacy Ludum Dare website, you can now find the code on GitHub:
Again, this is the legacy, not the new website. 7 glorious years of duct-taping features to WordPress. The workflow is very unsafe, and not 3rd party friendly. We’ve been committing and checking-out changes directly on the live website the entire time. Any time we made a change, we risked bringing down the website with a PHP error. Uncool. We used to host it on Google Code using SVN, but it’s now on GitHub. As of Thursday, I’ve committed the first real update since migrating the code GitHub. The workflow is just as unsafe, but a little more nerdy now as I commit changes using GIT, and check-out on the server using SVN (thanks to the awesome GIT->SVN feature built in to GitHub).
All aboard the Starship
Like I mentioned, development of the new website takes place on GitHub. About a month ago I started publishing my changes publicly. Eventually, you’ll be able to get the website code from here:
At the moment, we’re very much pre-alpha on the new website, so I don’t want to publish to that account yet. To me, github.com/ludumdare is for the live website, and at the moment the website isn’t live.
I’m still doing a lot of engineering, so instead, I’ve been keeping code in my personal GitHub account:
Once we are live, the Ludum Dare GitHub account will become active.
DairyBox: The Ludum Dare toolchain (and LD pun)
But again, since it’s pre-alpha, the download is on my personal GitHub account:
DairyBox is a Vagrant box based on Scotch/Box. It’s a VM that lets you run your own mini Ludum Dare server locally. This is a HUGE step up from the old development workflow. We can actually test now without bringing the server down! 😀
To use it, download and install DairyBox using GIT, and follow the instructions for cloning the source code (NOTE: It’s not a straight-up clone, it requires some git init trickery).
I want to help! What should I do?
At the moment, a lot of things are broken, unfinished, undocumented, and a little weird, but it is open now.
I’m not really expecting anyone to start sending me pull requests yet, but feel free to surprise me. As I see it, I still have a lot of core things to finish (and explain). There’s a method to the madness, and hopefully the rest of this post will give you broad idea of what’s going on.
There’s a very early effort in to auto-documenting the code. You’ll find a few Doxygen, JSdoc, and APIdoc comment blocks in various parts of the source code, but not enough since a number of things are still in flux. Scripts for documentation generation are in
/scripts/docs/, and will eventually be hosted on http://dev.ludumdare.com.
Bugs reports (though admittedly it’s a bit early for testing), for now post them to my ludumdare issues page.
Show me your ship!
Alright! If you want a look, here’s a sneak peek.
IMPORTANT: The website is in EARLY DEVELOPMENT. Many things don’t work, and many more don’t exist yet. It is not pretty to look at. It’s about function, not style.
You have been warned
This is a live snapshot of what’s currently in my GitHub repo. At times it may be out-of-date, it may be broken, but it’s real.
And just to mention it, you’ll note that URL uses HTTPS. There’s still work to be done on this front, but moving forward, we’ll be doing the security thing right.
Is that it?
Of course not.
What you don’t see is that Ludum Dare is now broken up in to several smaller modules. I’ll talk about some of the modules in detail in later sections.
But first, one of the main modules, the Static Website:
LD Module: The Static Website
It’s not really something you can look at, but the WIP static website is over here:
/www/public-static/ and the symlink
The URL to the static website will change, but basically, the static website is part optimization, and part future proofing when we need to scale-up by cutting up the server.
Today, the main dynamic element of the static website is the Image Resizer.
The image resizer is the thing that resizes and crops the images and avatars you upload. We did various things in the past, but the new resizer is designed to be less wasteful and take advantage of the CDN. When I talk about the 21 Gigabytes we currently use, a few of those GB are extra copies of images resized to fit some dimension. In the grand scheme of things, omitting those extra files doesn’t save much, but for a few minutes the resized images will live in RAM, long enough for the CDN nodes to grab a copy and cache them around the world. Not to mention, storing them in RAM saves us a disk access, so we can feed all those requests faster.
You can see a demo of the image resizer on wip.ludumdare.com. That unusually large cropped chicken image? Yeah, that’s the resizer’s handywork.
We are still not going to host downloads of your games, but we will be looking in to making things easier when it comes to hosting your downloads and source code.
What’s new with Editing?
Lots! Here are some highlights:
- All text is now written in Markdown instead of HTML (GitHub compatible)
- Posts, Pages, Games, Comments, everything you’d want to write a paragraph of text for will share the same editing interface (because internally, they’re are the same).
- Emoji and Short Codes. All the
:blowfish:you could ever want, just a few keystrokes away (see emoji.codes)
- AJAX. Editing is now inline. Many things should work a lot faster (since there’s less data to send).
- PATCHING: This is a big one. Every piece of content on the website can be edited by anyone. Patches are then sent to the original author, or team responsible. The changes can then be approved with a click by author, or a member of the Editor Team.
You can try a very rough version of the Patching, Inline Editing, and Markdown with Emoji and Syntax Hightlighting workflow on the WIP website (NOTE: Changes are not saved, and there is no notice when you’ve viewing a patched version of a post).
More things that are coming:
- @mention people by user name to send them a notice, and link to their user page.
- @mention groups such as the Help Team (@help, users that like to help people and answer questions), Editor Team (@editor, users part of our patch approval and post moderation team), the Press Team (@press, anyone crazy enough to listen to requests to check out a game), and so on.
- Game Control Short Codes. This is a project I’m going to need an artist to help with. Like the Emoji, I’d like us to have a library of game controls. Things like
:key_wasd: :key_arrows: :key_space:and many more for describing your game control using the keyboard. Also, mice, gamepads, and more exotic controllers like microphones, MIDI interfaces, gestures cameras, and so on.
- Short Code auto-completion. Just start typing an emoji code or a game control code, and get a pop-up of possible choices.
- Smart Embedding of External things. For example, YouTube and SoundCloud links auto-embed, perhaps Twitch and other live streaming services too, Tweets, and really, anything that makes sense.
- Smart Embedding of Internal things. Paste a link to a Game or Tool, and we auto-insert a little mini-widget. It’ll probably make sense to do it for other internal types like Events, Users, maybe Tags and Platforms as well. In some cases there will be variant URLs (i.e. /user/myname/game) that shows a personal game list instead of your personal info. If it makes sense, and there’s something worth showing (beyond a basic hyperlink), we’ll probably do it.
What’s new with submitting games and things?
Oh boy, here we go:
- Though not actually required, you will be encouraged to sign up for an event before it starts.
- Tags. Is your game a 5-Player Local-Multiplayer One-Button Platformer with RPG-Elements and a Colorblind-Mode? We got that.
- Tools. Made in Unity, RPG Maker, and Pygame, using Tiled, Photoshop and GoatTracker
- Platforms. Okay, we had platforms before, but now we can be a little more specific. Also…
- Find games by Tags, Tools used, and Platforms: Yes finally, I’ll be able to tell you exactly how many Unity games there are. And not only games in an event, *total*, for all events! More usefully, as we start collecting the data, we’ll be able to start tracking trends and the popularity of tools within the community.
- Submit Post-Compo (Post-Jam) Games: Did you remake/redo your game after the event, and publish it somewhere? You can add it!
- Submit “Late” Games: Did you not finish on time? Did you make a game inspired by a Ludum Dare theme from 5 years ago? You can add it!
- Add Games from Other Game Jams: Do you 1GAM? Do you Global Game Jam? Great! You can add those too!
- Add MORE games: You’ve added all your Jam games now, what else have you made?
- New Submission Types. For future Ludum Dare stuff, we’ve added support for several non-game submission types. Again, this is support for them! This doesn’t mean Ludum Dare is accepting these sort of submissions (not yet anyway).
- New Submission Type: Demos and Intros. Do you rock the demoscene? Great, we support those too! ALSO, alongside some of the bigger game jams, we’re adding a library of popular and historic Demoscene events. Assembly, Revision (Breakpoint, Mekka & Symposium), etc. You tell us what we need!
- New Submission Type: Media. Do you make videos about Ludum Dare games? We want to catalogue them!
- New Submission Type: Crafts. A catch-all for everything else. Did you bake a cake inspired by the theme, or a little comic book?
Much of this is the ground work for future features.
- Where it makes sense, I’d like to make it easy to import games from some of the bigger Game Jam sites.
- Smart Embedded Audio and Video Media Submissions: Support for SoundCloud, YouTube, and whatever other media services make sense.
- Picture and Paged Craft Submissions: The idea behind Craft Submissions is they’re things that may not necessarily have a download (but one can be provided). These things will have to be shown off with Pictures and Photos. Just in case paged comics become a thing, we can look in to adding arrow keys or touch gestures to flip pages.
- This isn’t a feature per se, but I would like to work with the various online game hosts like Newgrounds, Kongregate, GameJolt and Itch to see if we can find a way for us to directly embed games hosted on these services. I’m not that far yet (gotta finish the site first), but I want to start talking with folks about this.
- Similarly, we might be able to streamline uploading directly to game hosts if we can rig up some magic
- I don’t know what to do with it, but there’s always the possibility of a deeper user integration with GitHub. I’m not sure what we’d do with it (generate a wacky graph of how active you were over the weekend?), but they have an API. Feel free to suggest things.
- Custom Events: Run your own Jam, Demoparty, Bake-off, whatever!
We don’t have an ETA on Custom events, but it’s on the roadmap. It’s probably one of the last things on this list, since there are perfectly good other services for hosting your Game Jams today.
What about an API?
Oh yes, there will be an API.
On the topic of adding other Jam games, I’ve published a specification for an API. I’m calling it the EasyJam Spec.
Effectively it’s a less complicated OAuth (which you could just cheat at, and use usernames for Tokens). I’ve tried to be inclusive of how some of the other Jam websites work, but feel free to tell me if I’ve neglected anything.
Initially, EasyJam is about synchronizing game data between Jam sites. It’s opt-in, and designed to be used securely. You’re not required to use it securely, and details on how do so are mentioned.
Ludum Dare will have a secure implementation of EasyJam, and I’m hoping other sites decide to join us.
What about Teams?
Oh yes, there will be team support.
One of the reasons why “signing up” is now a thing is because that’s how you form teams.
There are actually 2 types of teams:
By default, Teams are one-shot, for a single event only. This is because the people on a team will sometimes change even if the team name doesn’t. We’ll look in to making it easy to clone a previously used team when you participate in the latest events.
Teams have a Captain (the team author), and it’s their responsibility to manage the team, add members, and handle the submission. You’ll also be able to add other Captains to your team.
As a member of an team, your posts will be marked in some way to showcase this affiliation (How we’ll do it is TBD). And when you view the team page, you’ll see a list of posts from all members of the team. Game pages will list all members of the team that created it.
Aside from Teams, there are also Groups.
Groups are stricter in how they work. Initially, there will only be a few Groups, mainly internal ones used to manage permissions, and a few support groups (@help, @editor, @press).
Eventually there will be a process for vetting and approving Groups, but understand this:
Groups WILL NOT be able to submit entries to Ludum Dare events.
Groups are a bit like Subscriptions. As a member of a group, you get notifications to that group.
Groups can be ether public (anyone can join), or private (members must be added manually).
We may eventually open up support communities (Groups) for popular tools. @unity-help, @blender-help, @gimp-help, @photoshop-help, etc. The popularity of Tool Pages will likely play some role in vetting support communities. Like @help is for general Ludum Dare support, help communities will be open for anyone to join, to assist other users with the issues with their respected tool.
And where it makes sense and is useful, we’ll look in to adding other groups. In some cases, it’ll make sense to give ownership of content to a group (for example, if we have an @unity group of Unity Employees, it makes sense that they can edit their own Tool page).
But again, I’d like to keep the number of groups low initially. I want us to feel-out where they are useful.
What about Notifications?
Okay yes, up until now we’ve more or less assumed we support Notifications. We do. It’s the glue that binds together a number of features.
Initially, notifications will be limited to comments (finally, letting you know when someone comments on your game), and the usage of @messages to People and Groups.
Joining a Group is a bit like subscribing to notifications. If you’re a Captain, you’ll be able to control pages associated with your team. If you’re a member, you’re just there to listen.
Eventually we will add private messages, but that’s a low priority. There are plenty of ways to contact people privately outside Ludum Dare, so I’m sure you’ll figure something out.
What’s new with the post feed/timeline?
So the feed needs work. Anyone that’s tried to watch the stream during events knows how chaotic it is. Practically every minute there’s a new page of posts, and some people don’t post since they know it wont stick.
Plus, there’s a good reason to consider adding other content besides Posts to the timeline. But before we add more content to the feed, we need to fix it.
So, soon we’ll be:
- Splitting the Post Feed
Every user will be assigned to Sub-Feed of the main feed.
I’m calling these sub-feeds Leagues. Admittedly, that’s an odd name choice, but in the future I’d like us to have some fun with the split. My thought is that eventually, we may treat the splits like a bit of a sporting team, country (but not actually a country), or a “House” in Harry Potter. We want a diverse group of people in each group, because that’s interesting. We want interesting.
As it is today, things move too fast. You can’t have a discussion during Ludum Dare, because the post asking the question is gone right away. But if we slow things down a bit (by having less content), you’ll be able to get an answer.
So now that we’ve split the feed, what about cool stuff, or people I like?
- Post Love is Back! Now, Post Love will be the way good posts cross the League barrier. When enough people in a League give it +1’s, it’ll show up for all leagues. That way, you don’t miss popular posts from outside your League.
- Follow Users. You’ll be able to follow your favourite users in the Ludum Dare community, so no matter what League you’re in, you’ll see their posts.
- Published Games in the feed. As soon as you click that Publish button on your game, like a post, there’ll be a brief notice in your Leagues timeline. The name of your game, and a banner grahic or trimmed screenshot. They wont be too large, a bit bigger than a tweet. There is a very brief sample of this behaviour on wip.ludumdare.com. I haven’t done any styling to them yet, just shown what it would be like.
Things we’ll be looking in to:
- League Names. We need to figure out the granularity needed for the public timeline first. Once we know what a comfortable number of users is, we’ll look in to naming the sub-feeds. Then you can say things like “I’m part of Kitten League” or “Zombie League”, fun things like that. It’ll have no bearing on results (we shut-down the leagues once the event ends), but during the Pre-LD hype, I’d love to see playful rivalries between the Leagues. 😉
- Multiple user signups as a single timeline post. Not necessarily a replacement for “I’m In” posts (which aren’t really necessary, but add character to the pre-LD hype). Rather than adding a bunch of individual “who is in?” blips to the timeline, we can group them together in to little blobs. The point of these is that you might see people that you want to follow.
- AJAXy/WebSockety Twitter-like auto-updates. Rather than reload the page constantly, we’ll look in to notifying you when there are new posts. It probably wont be as aggressive as Twitter (every 10 seconds), but we’ll figure out what works.
- AJAXy/WebSockety Theme Announcement. Same as above, we’ll look in to a smart way of announcing the theme on the website that doesn’t bring the server down.
- External Posting. I.e. post to your account from Twitter, Facebook, Tumblr, Instagram, or your Blog (RSS). Not everybody likes to log their progress on the Ludum Dare website.
What’s new with user pages?
- As mentioned above, a chronological list of MORE than just your Ludum Dare games. Games from Other Jams, Demos, Media, Crafts (as soon as we add support for adding them)
- Any Events you’ve run (MiniLD’s, Custom, etc)
- Published Games intermixed in your summary timeline, and post timelines.
- Some new types of Navigation. View your Posts, your Favourites, your Games, who you Follow, etc.
- New Module: Jammer
I’ll talk about Jammer in a moment.
Here are a couple test user pages on the WIP. Again, try not to pay much attention to what they say or how they look, just that they exist.
Things that we’ll be looking into over time.
- Featuring of your favourite games (they appear first)
- Ludum Dare Submission Stats. Graph showing how your submissions have done over all the events you’ve participated in
- Other Graphs. As we figure out other sensible data people will want to see, we can add more personalized graphs
- The Trophy Rack. This is going to be a bigger project, totally silly, but I’m all about fun. I’m definitely going to need an artist involved. I’d like there to be a real digital trophy graphic for each of the Ludum Dare categories. If we’re feeling really silly, we might actually build an HTML5 Canvas app with 2D physics that lets you arrange your trophies, and save the layout. And if we’re feeling reeeeeealy silly, we can add breakpoints to the Trophy geometry, so you can smash your trophies in a fit of rage (and repair them with a click, though it would be funny if you couldn’t repair them).
- User Trophies. This will probably come much later. Rather than the 64×64 images we used to use for Trophies, I think I’d rather we have a library of miscellaneous trophies you can pick from, and a pick a color. That’s my thought at the moment, but if we come up with a nice way to let you generate them, then maybe. Also, User Trophies will be assigned to a specific thing (a game usually), instead of a user like we did in the past. User specific trophies will still be supported though.
- The Cardboard Box. Completing the Trophy Rack metaphor, a place for your “Coolest Haircut” trophy. Basically, the things you don’t want to show-off on your Trophy Rack. Nobody ever throws away their trophies, they just live in a box. 😉
- Trophies on Featured Games. As we figure out the layout, we’ll find a place to put those important trophies next to the game banner. We might only be able to fit a couple in this view, so you pick which ones matter most to you.
Jammer, we should talk about Jammer.
LD Module: Jammer
Jammer is an extension of your Ludum Dare user page.
You’ve used Ludum Dare before. We put weird stuff in the sidebar, big noisy banners and bars to get your attention, a movie player, potatoes, yadda yadda. It can be nuts, but that’s what Ludum Dare is. This frantic sort of crazy event.
Jammer strips that away. It’s all about you. Just you and your games.
The website (NOTE: link is not active yet):
Where ‘user-name’ is your ludumdare.com user name. Jammer is your user page, without the noise.
Sure, Jammer isn’t for everyone. If you run your own portfolio website, you probably don’t need. But if you’re an active Ludum Dare user and game jammmer, we give you a nice easy to remember URL for you and your stuff. No need to explain to people what a Ludum is, nor debate Dare’s correct pronunciation. And heck, it’s a simple enough URL to tell someone aloud, with your actual human voice. 😉
That’s Jammer in a nutshell.
As time goes on, we’ll give you more and more flexibility for customizing your page. Here are a couple WIP links. Please excuse the lack of formatting and style. When it’s done it wont say silly things like “users home page”. It will be styled in a nice way with banners of your games (not text links). At the moment it’s also a bit of an ugly testbed for the Markdown decoder, so try not to criticize what you’re seeing too much, just that it’s there and using the same data as user pages. 😉
You’ll note on my page (
/pov), I’ve customized the colors, and I added my Real Name. This is done through some Jammer specific metadata we assign, just to show that customization support is coming.
LD Module: ldj.am
This is a minor module I thought I’d mention. It’s doesn’t really have anything to do with Jammer, but it’s loosely related, in that it also deals with URLs.
ldj.am (LDJAM) will be a URL shortener for the main website.
URLs will look something like this (NOTE: link is not live):
This is in contrast the clean URLs of Jammer, these are designed to be small.
Everything on the New Ludum Dare website has a Unique Id. Posts, Games, even Users and Events. So that crazy string (4cDfm) is a packed version of the internal “Node Id”. For the sake of example, lets say it’s 332,511.
The encoder is something that needs to be tuned. It’ll be close to a 48 bits per character. It’s a combination of the numbers 0-9, uppercase and lowercase letters. If you were to total that, you’d get 62 characters, a mere 2 short of 64 (or base64 encoding). The problem with base64 is there’s a risk of generating profanity and offensive things. So to avoid ldj.am/shitface, some reoccurring characters in English profanity are removed from both upper and lower case. And for legibility, a few very similar characters can be removed (such as S instead of 5, l instead of 1).
To decode, we first decode the packed value. We lookup the Node Id in the database, then we have to recursively look-up the slug-parts of the URL by walking the tree backwards until we hit the root node. All the parts together gives us the true Ludum Dare URL.
Of course, we can do some smart, optimal things when it comes to the reverse lookup. We can cache the slugs of the Hierarchy Nodes by Id in RAM. Even with all the non-Ludum Dare events, there shouldn’t be more than a few hundred of them. That said, this is probably one of the main bottlenecks for Custom Events, ’cause as soon as we open this up, the cache size is no longer in our control (unless we explicitly ignore the custom tree, and just do an extra lookup).
Anyway, once we know our true URL, we emit a redirect (302) to that.
So the ldj.am URL shortener will not be general purpose. It’s specifically for shortening ludumdare.com URLs.
And like everything else, the WIP version is over here:
So when it comes to User pages, you’ll be offered both an ugly URL (
ldj.am/b382u) or a clean URL (
jammer.bio/johnjoe25). The ldj.am URL will go directly to your Ludum Dare user page, and Jammer URL directly to the clean less-noisy Jammer version of your page.
That’s a lot of stuff. There’s more, but some of these things just aren’t as cool.
- Security. Probably the broadest and most complicated one-word subject there is.
- Logging in and out. We gotta be able to do that.
- Account creation and activation. That whole e-mail exchange thing.
- Password Reset. Yep.
- HTML to Markdown filter. We’ll best-guess the intentions of HTML tags in imported posts.
- Reserved Names List. We don’t want people camping names, or doing things that will cause confusion. So no, you wont be able to call yourself Microsoft. But if a few Microsoft employees join the community, and it makes sense that they have a group name, they can contact me and we can set them up.
- Things I can’t remember.
This is Starship in a nutshell.
What about Event Features?
You’ll note that I’ve only covered User specific features so far. I figure this post is long enough, and at the moment my focus is on User features. Most people care about how Ludum Dare changes for them, not how it changes for me.
As things get further along, I’ll write another post detailing more of the internals of how events run, the job queue, event stage automation, and so on.
That’s all for now. Congratulations, you made it to the bottom.