RTFM! (or: Why every web developer should read RFC 2616)

As a software developer you always have to deal with users who find it difficult to use your software, or may -in one way or another- find it challenging to understand the interface or the functionality you’ve designed.
But that’s okay, because we are geeks who spends most of your adult lives behind a computer screen, building software and applications that -to us- are interesting and make perfect sense.
The reality is, without Graphic Designers to make our ugly interfaces sexy, and without Marketers to put together a clear message that explains what our thousand lines of code does, and without Sales Gurus to make these lines of text sell for $$$ we’d all be the laughing stock of the rest of the world, with our crazy, ugly looking products that do what any capable human being can do.

Try telling Don Draper back in the 50s that we could one day replace his secretary with a voice enabled application on a portable device that you can put in your pocket and operate it simply by touch it’s glass surface. oh and it will be called Siri.

He’d probably punch you square in the face.

So I’m never upset, offended or at the least, bothered, by any feedback or criticism from people using software that I’ve build, I’ve always found it to be an enlightening and humbling experience to see into the way other people think and react to technology that I’ve build.

But! those are people who are not developers, software engineers or in any way shape or form, familiar with the inner workings of the technology they use day to day.

I’ve always written software for people like that, and I always will. recently however, I created REST Console an HTTP Client for Google Chrome. this is an application directly targeted towards software developers, not your average consumer.

Software developers (in my mind) are people who can create code in the form of software and applications, much like the way artists create art.
Most of us don’t understand how the artist creates his art, we don’t really know what the tools they use are beyond our basic knowledge or a paint brush. But we all appreciate their work, some of us love, some don’t.
Software is the same as Art, many people don’t have the skill for it or the understanding of how it works, but they use it every day.

Ever since REST Console v.4.0.0 came out, I’ve been getting emails, comments, and github issues telling me its broken.

well, Its not. you are using it wrong!

Too aggressive? how about this: You don’t know how the internet works!

I find it utterly frustrating and completely shocking that the target audience for this particular software don’t know how to use it!

I figured Software Developers of all people need the least explanation, hand holding, or even guidance in using a software application that is SO specific in nature to what they do every day as part of their daily jobs (or hobbies, whatever) that I didn’t need to include a manual, or even do much explaining how the app works, they can figure it out themselves, they have the skills, they build apps like this every day!. Or at the very least, Google for some background information on the solution the application provides.

v4.0.0 of REST Console was a major re-write, but other than the interface and the code, little has changed or was added, except for one little tiny thing:
Header fields are no longer pre-populated with a default value.

This has apparently caused HAVOC.

It seems a significant number of Software Developers out there don’t really understand how the internet works, well, allow me to shine your way to a better understanding…

See, there is something interesting out there called RFC 2616 that, in simple terms: defines how HTTP requests are made over the internet.
and HTTP Requests are exactly what REST Console is built to do.

People were complaining that v4.0.0 is broken, why? because their POST payload is not being received on their servers.
It seems the mistake these developers have done (beyond not reading RFC 2616) is forgetting to set their Content-Type header before they send their POST requests.

This alone is not a big deal, I started telling people they may have forgotten to set their Content-Type header, and they proceeded to check the box to enable it and come back and complain that it still didn’t work.

Turns out none of these guys realized that they need to set their Content-Type to “application/x-www-form-urlencoded” which is the Internet media type for sending HTML Form Data, something they would have known have they read RFC 2616

RFC 2616 actually has no mention of “application/x-www-form-urlencoded”, it only references RFC 1867 regarding Form-based File Upload in HTML, which mentions “application/x-www-form-urlencoded” which is actually defined in HTML 4.01 Specification, Section 17.13.4.1 only to discuss its limitation and being: “inefficient for sending large quantities of binary data or text containing non-ASCII characters.” and suggests using “multipart/form-data” as it accommodates a wider spectrum of data to be transferred over HTTP.

I find it actually scary that some developers don’t know that.

REST Console is a tool to build HTTP requests, not emulate an HTML form. it’s that simple.

PHP (and other languages) expects the Content-Type header to be set to “application/x-www-form-urlencoded” so it would parse the POST url-encoded string into a $_POST variable … but that’s not how all HTTP requests are made.

While this is a “typical” behavior for sending data through HTML forms, you can in fact, send a payload of a url-encoded “key/value” string and tell the server its “application/xml”!

Its up to the HTTP client library to determine what to set the Content-Type header to, and to the server to determine how interpret the request body, based on the Content-Type header it receives …

I feel really frustrated writing this post, I shouldn’t have to, this should be common knowledge for people in our field, but there it is. and it scares me.

Based on how many people have complained about POST data not being sent, maybe I’ll make the Content-Type header pre-populated again, This post will not stop the questions, the bad reviews in the Chrome Web Store or the github issues. There are already a significant number of them, and people didn’t bother reading the replies I made. so why should this post help?

The one thing I’m now almost certain people will never do, is read RFC 2616

Please prove me wrong.