I was inspired to write this How-To article because, sometimes, in their API documentation, API providers fail to tell you what sort of data their APIs return and how that data is formatted. Yes, I know. It's hard to believe how, more than a decade after Web APIs first hit the scene, API providers still omit this all-important detail. But, it happens. So, this article is a quick introduction -- especially for those who are new to working with APIs -- on how to discover this basic bit of information. It should be fun for anyone -- especially beginners -- looking to dip their toes into the world of Web APIs. One note: this is not a comprehensive overview. It's just a very quick and dirty hands-on exercise to get your feet wet (hands, feet, toes? OK, enough of that!).
When any software -- your Web browser for instance -- communicates with a Web server, that communication typically starts with a collection of information known as the HTTP header that gives the other end some idea of what's going on. It's called an "HTTP header" because communication of this nature with a Web server typically happens over the HTTP protocol (hypertext transfer protocol); the official protocol of the World Wide Web. In fact, if you wanted, you could also call a "Web server" an "HTTP server." The word "header" naturally means the head-end of the communication, or what comes first. One of the fields of data that's typically included in an HTTP header is the "content-type" and, confusingly, the information that's provided in that field is referred to as the "media type." If you're wishing they just kept the language consistent, well, me too. The content-type field in the HTTP header is the field that gives you some idea of the way that the data returned by an API (or any Web URL) is formatted.
So, let's take a look at an example involving an actual Web API. By the way, one reason they're called "Web APIs" is that, in the same way that a Web server will send a Web page to your browser when your browser visits that Web server (aka "Web site"), that same Web server might send a differently formatted response when some other software like a mobile application visits its API. In fact, this is where the content-type field in the browser plays a big role. Either way, the information is being delivered to the consumer of that data (your browser, a mobile application, etc.) over the Web's protocol; HTTP.
Let's take compare and contrast some example responses from one Web site: the Weather Underground (important note: the Weather Underground is NOT one of those sites whose documentation does a poor job documenting how its API formats its data. It actually does a pretty good job.)
Even though it's the root domain (the ".com" part) of the Weather Underground's API, if you visit http://api.wunderground.com with your browser, you'll probably get a Web page that's the same as Weather Underground's home page. Like any response from a Web server, this Web page was preceded by an HTTP header and if you use the following cURL command from the terminal app on your Mac or Linux machine (or using one of many cURL utilities you can download and install on Windows) to visit the same URL, you'll see what the header looks like:
curl -I http://api.wunderground.com/The -I option (that's an upper-case "i") is interchangeable with the "--head" option and tells cURL to fetch the just the header from the specified URL. In the case of http://api.wunderground.com, the output will look something like this (to keep it simple, I've redacted some cookie data):
HTTP/1.1 200 OK
Server: Apache/2.2.15 (CentOS)
Access-Control-Allow-Origin: http://www.wunderground.com
Access-Control-Allow-Credentials: true
X-CreationTime: 0.156
Content-Type: text/html; charset=UTF-8
Expires: Mon, 29 Aug 2016 12:40:52 GMT
Cache-Control: max-age=0, no-cache
Pragma: no-cache
Date: Mon, 29 Aug 2016 12:40:52 GMT
Connection: keep-aliveNotice the sixth line down? It says:
Content-Type: text/html; charset=UTF-8And that makes perfect sense, doesn't it? After all, if you visit that URL with your Web browser, you get a Web page back (and a Web page is a mixture of text and HTML; the markup language that makes Web pages look pretty). So, the content type is naturally "text/html."
But now, let's actually cURL the API with a real API request. To cURL the Weather Underground's API, you'll need an API key from Weather Underground. To get this, created a registered user account on wunderground.com. Then, visit the API pricing page and "purchase" an API for the Stratus Plan (don't worry, it's free). You'll have to fill out a form (don't worry, it doesn't have to be too official) and then, when your free purchase is complete, you'll see your API key (it'll be a long string of characters). This time, let's enter a slightly different cURL command:
curl -I http://api.wunderground.com/api/<<your API key goes here with no brackets>>/conditions/q/94107.jsonHere's what it might look like with an actual API key. This API key is fake so it won't work if you try to cut and paste it. Be sure to use your own:
curl -I http://api.wunderground.com/api/1b02bc3d1ab6c044/conditions/q/94107.jsonOK, in fairness, for the Weather Underground's API, we really don't need to go through this exercise because the URL itself ends with "json." It's a dead give away that the data that will be returned by this URL is formatted with JSON. But, we're just having fun, right? This time, the output will look something like this:
HTTP/1.1 200 OK
Server: Apache/2.2.15 (CentOS)
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
X-CreationTime: 0.154
Last-Modified: Mon, 29 Aug 2016 14:12:32 GMT
Content-Type: application/json; charset=UTF-8
Expires: Mon, 29 Aug 2016 14:12:32 GMT
Cache-Control: max-age=0, no-cache
Pragma: no-cache
Date: Mon, 29 Aug 2016 14:12:32 GMT
Connection: keep-aliveNotice the seventh line down? This time, the Content-Type field is different. Right? What does it say?
Content-Type: application/json; charset=UTF-8It's telling us that the data that will follow (if we didn't retrieve just the header) is formatted with JSON. Sure enough, let's take a look at the data you'd get if actually called the API. To do this, point your browser to the same URL you just fed to the cURL command and, in response, you should get a whole package of data formatted as JSON that looks something like this:
{
"response": {
"version":"0.1",
"termsofService":"http://www.wunderground.com/weather/api/d/terms.html",
"features": {
"conditions": 1
}
}
, "current_observation": {
"image": {
"url":"http://icons.wxug.com/graphics/wu2/logo_130x80.png",
"title":"Weather Underground",
"link":"http://www.wunderground.com"
},
"display_location": {
"full":"San Francisco, CA",
"city":"San Francisco",
"state":"CA",
"state_name":"California",
"country":"US",
"country_iso3166":"US",
"zip":"94107",
"magic":"1",
"wmo":"99999",
"latitude":"37.76834106",
"longitude":"-122.39418793",
"elevation":"5.00000000"
},
"observation_location": {
"full":"Potrero Hill, San Francisco, California",
"city":"Potrero Hill, San Francisco",
"state":"California",
"country":"US",
"country_iso3166":"US",
"latitude":"37.769039",
"longitude":"-122.402245",
"elevation":"20 ft"
},
"estimated": {
},
"station_id":"KCASANFR880",
"observation_time":"Last Updated on August 29, 7:23 AM PDT",
"observation_time_rfc822":"Mon, 29 Aug 2016 07:23:10 -0700",
"observation_epoch":"1472480590",
"local_time_rfc822":"Mon, 29 Aug 2016 07:23:14 -0700",
"local_epoch":"1472480594",
"local_tz_short":"PDT",
"local_tz_long":"America/Los_Angeles",
"local_tz_offset":"-0700",
"weather":"Overcast",
"temperature_string":"57.9 F (14.4 C)",
"temp_f":57.9,
"temp_c":14.4,
"relative_humidity":"83%",
"wind_string":"From the West at 6.5 MPH Gusting to 9.8 MPH",
"wind_dir":"West",
"wind_degrees":281,
"wind_mph":6.5,
"wind_gust_mph":"9.8",
"wind_kph":10.5,
"wind_gust_kph":"15.8",
"pressure_mb":"1019",
"pressure_in":"30.10",
"pressure_trend":"0",
"dewpoint_string":"53 F (12 C)",
"dewpoint_f":53,
"dewpoint_c":12,
"heat_index_string":"NA",
"heat_index_f":"NA",
"heat_index_c":"NA",
"windchill_string":"NA",
"windchill_f":"NA",
"windchill_c":"NA",
"feelslike_string":"57.9 F (14.4 C)",
"feelslike_f":"57.9",
"feelslike_c":"14.4",
"visibility_mi":"10.0",
"visibility_km":"16.1",
"solarradiation":"30",
"UV":"0.0","precip_1hr_string":"0.00 in ( 0 mm)",
"precip_1hr_in":"0.00",
"precip_1hr_metric":" 0",
"precip_today_string":"0.00 in (0 mm)",
"precip_today_in":"0.00",
"precip_today_metric":"0",
"icon":"cloudy",
"icon_url":"http://icons.wxug.com/i/c/k/cloudy.gif",
"forecast_url":"http://www.wunderground.com/US/CA/San_Francisco.html",
"history_url":"http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KCASANFR880",
"ob_url":"http://www.wunderground.com/cgi-bin/findweather/getForecast?query=37.769039,-122.402245",
"nowcast":""
}
}OK, we're almost done! Just for kicks, let's do one more experiment with the Weather Underground's API.
The Weather Underground's API is pretty flexible. Whereas some developers prefer to work with JSON-formatted data, other developers might prefer to work with XML-formatted data. The Weather Underground's API is flexible in that the data it returns can be formatted or "serialized" either way! If you're an API provider, this is a nice option to offer your developers. So this time, let's cURL a slightly different URL for its header:
curl -I http://api.wunderground.com/api/1b02bc3d1ab6c044/conditions/q/94107.xmlYou're response should look something like this:
HTTP/1.1 200 OK
Server: Apache/2.2.15 (CentOS)
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
X-CreationTime: 0.086
Last-Modified: Mon, 29 Aug 2016 14:32:45 GMT
Content-Type: text/xml; charset=UTF-8
Expires: Mon, 29 Aug 2016 14:32:45 GMT
Cache-Control: max-age=0, no-cache
Pragma: no-cache
Date: Mon, 29 Aug 2016 14:32:45 GMT
Connection: keep-aliveInteresting eh? If you look at the Content-Type, this time it says "text/xml"! Of course it does! The Weather Underground has programmed its Web server so that no matter what Web address you visit (with your browser, with cURL, with an API call from a mobile application, etc.), the HTTP header in the response gives you some idea of what to expect in terms of how the data is formatted. Now, let's visit the same URL with your browser just to confirm that it does indeed come back as XML (instead of JSON). You should see something like this:
<response>
<version>0.1</version>
<termsofService>http://www.wunderground.com/weather/api/d/terms.html</termsofService>
<features>
<feature>conditions</feature>
</features>
<current_observation>
<image>
<url>http://icons.wxug.com/graphics/wu2/logo_130x80.png</url>
<title>Weather Underground</title>
<link>http://www.wunderground.com</link>
</image>
<display_location>
<full>San Francisco, CA</full>
<city>San Francisco</city>
<state>CA</state>
<state_name>California</state_name>
<country>US</country>
<country_iso3166>US</country_iso3166>
<zip>94107</zip>
<magic>1</magic>
<wmo>99999</wmo>
<latitude>37.76834106</latitude>
<longitude>-122.39418793</longitude>
<elevation>5.00000000</elevation>
</display_location>
<observation_location>
<full>Potrero Hill, San Francisco, California</full>
<city>Potrero Hill, San Francisco</city>
<state>California</state>
<country>US</country>
<country_iso3166>US</country_iso3166>
<latitude>37.769039</latitude>
<longitude>-122.402245</longitude>
<elevation>20 ft</elevation>
</observation_location>
<estimated>
</estimated>
<station_id>KCASANFR880</station_id>
<observation_time>Last Updated on August 29, 7:26 AM PDT</observation_time>
<observation_time_rfc822>Mon, 29 Aug 2016 07:26:05 -0700</observation_time_rfc822>
<observation_epoch>1472480765</observation_epoch>
<local_time_rfc822>Mon, 29 Aug 2016 07:26:12 -0700</local_time_rfc822>
<local_epoch>1472480772</local_epoch>
<local_tz_short>PDT</local_tz_short>
<local_tz_long>America/Los_Angeles</local_tz_long>
<local_tz_offset>-0700</local_tz_offset>
<weather>Overcast</weather>
<temperature_string>57.9 F (14.4 C)</temperature_string>
<temp_f>57.9</temp_f>
<temp_c>14.4</temp_c>
<relative_humidity>84%</relative_humidity>
<wind_string>From the WSW at 5.8 MPH Gusting to 7.4 MPH</wind_string>
<wind_dir>WSW</wind_dir>
<wind_degrees>243</wind_degrees>
<wind_mph>5.8</wind_mph>
<wind_gust_mph>7.4</wind_gust_mph>
<wind_kph>9.3</wind_kph>
<wind_gust_kph>11.9</wind_gust_kph>
<pressure_mb>1019</pressure_mb>
<pressure_in>30.10</pressure_in>
<pressure_trend>0</pressure_trend>
<dewpoint_string>53 F (12 C)</dewpoint_string>
<dewpoint_f>53</dewpoint_f>
<dewpoint_c>12</dewpoint_c>
<heat_index_string>NA</heat_index_string>
<heat_index_f>NA</heat_index_f>
<heat_index_c>NA</heat_index_c>
<windchill_string>NA</windchill_string>
<windchill_f>NA</windchill_f>
<windchill_c>NA</windchill_c>
<feelslike_string>57.9 F (14.4 C)</feelslike_string>
<feelslike_f>57.9</feelslike_f>
<feelslike_c>14.4</feelslike_c>
<visibility_mi>10.0</visibility_mi>
<visibility_km>16.1</visibility_km>
<solarradiation>37</solarradiation>
<UV>1.0</UV>
<precip_1hr_string>0.00 in ( 0 mm)</precip_1hr_string>
<precip_1hr_in>0.00</precip_1hr_in>
<precip_1hr_metric> 0</precip_1hr_metric>
<precip_today_string>0.00 in (0 mm)</precip_today_string>
<precip_today_in>0.00</precip_today_in>
<precip_today_metric>0</precip_today_metric>
<icon>cloudy</icon>
<icon_url>http://icons.wxug.com/i/c/k/cloudy.gif</icon_url>
<forecast_url>http://www.wunderground.com/US/CA/San_Francisco.html</forecast_url>
<history_url>http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KCASANFR880</history_url>
<ob_url>http://www.wunderground.com/cgi-bin/findweather/getForecast?query=37.769039,-122.402245</ob_url>
</current_observation>Do you see the differences between the JSON and XML versions of the same data?
OK, there you have it. A verty quick and dirty introduction to inspecting Content-Type's using the cURL command.