JSON is a lightweight data-interchange format. It was formally standardized by Douglas Crockford, and since has been received almost universally as a simple and powerful representation of data for transmission between two entities, regardless of what computer language those entities run in natively.
The same-origin policy in browsers dictates that certain types of data transfer in the browser layer (via JavaScript) must be restricted to only occur if the target resource's domain is identical to the page making the request. This policy is in place in all modern browsers to help protect users from unwanted or unsafe malicious JavaScript behaviors.
Cross-domain Ajax refers to the idea of making requests across domains in opposition to the same-origin restriction. However, cross-domain Ajax is not inherently unsafe or evil -- it is in fact essential to many of the world's most popular and useful mashups. But cross-domain Ajax done properly seeks to make such communication using techniques which, for various reasons, are not subject to the same-origin policy.
One such mechanism which can request content cross-domain is the <script> tag. In December 2005, Bob Ippolito formally proposed JSONP (later dubbed JSON-P, or JSON-with-padding) as a way to leverage this property of <script> tags to be able to request data in the JSON format across domains. JSON-P works by making a <script> element (either in HTML markup or inserted into the DOM via JavaScript), which requests to a remote data service location. The response (the loaded "JavaScript" content) is the name of a function pre-defined on the requesting web page, with the parameter being passed to it being the JSON data being requested. When the script executes, the function is called and passed the JSON data, allowing the requesting page to receive and process the data.
Example:
function handle_data(data) {
// `data` is now the object representation of the JSON data
}
---
http://some.tld/web/service?callback=handle_data:
---
handle_data({"data_1": "hello world", "data_2": ["the","sun","is","shining"]});
As you can see, the remote web service knew what function name to call based on being told the function name in the URL that made the request. As long as that function is in fact defined in the requesting page, it will be called and passed the data it was asking for.
Thus far, JSON-P has essentially just been a loose definition by convention, when in reality the browser accepts any abitrary JavaScript in the response. This means that authors who rely on JSON-P for cross-domain Ajax are in fact opening themselves up to potentially just as much mayhem as was attempted to be avoided by implementing the same-origin policy in the first place. For instance, a malicious web service could return a function call for the JSON-P portion, but slip in another set of JavaScript logic that hacks the page into sending back private user's data, etc.
JSON-P is, for that reason, seen by many as an unsafe and hacky approach to cross-domain Ajax, and for good reason. Authors must be diligent to only make such calls to remote web services that they either control or implicitly trust, so as not to subject their users to harm.
There are many alternatives to JSON-P, but each of them has their own drawbacks or challenges. These techniques will not be covered in detail here, except for one: CORS (aka, "Cross-Origin Resource Sharing") -- the most recent addition to browser JavaScript for making cross-domain Ajax calls. Put simply, CORS is an extension to the standard XMLHttpRequest (aka, "XHR") object, which allows the browser to make calls across domains (despite the same-origin restriction). But it does so by first "preflighting" a request to the target server to ask it for permission.
In other words, the remote server is able to opt-in or opt-out of such communication as it sees fit. For instance, a server may expose some content to Ajax requests from only a pre-defined list of approved site domains, and reject all other requests from any other pages. Or, a server may open up its content to be retrieved by any other domain, if it sees fit to do so.
At first glance, it may seem like CORS is an ideal solution for cross-domain Ajax, and makes "hacks" like JSON-P obsolete. Nicholas Zakas recently wrote about CORS for cross-domain Ajax, and gave it a glowing endorsement as the future of cross-domain Ajax in browsers.
Whether CORS as the "standard" will end up being widely implemented and relied upon for broad cross-domain Ajax usage on the web remains to be seen. There are some drawbacks, of course, as the devil is always in the details.
Firstly, CORS requires a server that implements a web-service to implement some non-trivial logic to intercept request-headers in the special "preflight" authorization requests, and to respond with properly formatted response headers depending on the server's intended policy response. If a JSON web-service needs to support cross-domain requests with JSON-P, it's a fairly straightforward change to wrap the JSON piece in a simple function call.
But asking web services across the internet to all implement lower-level logic at their web server layer to send and receive special custom headers is possibly a bit more tricky, depending on what web server software and permissions are in place. It may take several years for this technique to catch on and be to the point where the majority of web service providers is CORS-compliant. As of now, it's very new, and very few web services have done so, as opposed to tens of thousands of existing JSON-P enabled web service endpoints.
Moreover, CORS was implemented (albeit with some slight syntactic differences) only as of IE8, and not yet at all in Opera (as far as is known). So, there's a pretty decent chunk of web visitors whose browser does not support CORS, which means that a fallback for alternative cross-domain Ajax is still going to be necessary for the mid-term (1 to 3 years, perhaps).
For now, JSON-P is a viable solution for cross-domain Ajax. While CORS may represent a more direct and less hacky way of such communication, it should probably be deployed in tandem with JSON-P techniques, so as to account for browsers and web services which do not support CORS. However, the safety concerns around JSON-P are valid and should be addressed.
So, a stricter subset definition for JSON-P is called for. The following is the proposed "standard" for only what should be considered valid, safe, allowable JSON-P.
functionName({JSON});
obj.functionName({JSON});
obj["function-name"]({JSON});
The intention is that only a single expression (function reference, or object property function reference) can be used for the function ("padding") reference of the JSON-P response, and must be immediately followed by a single ( ) enclosing pair, inside of which must be a strictly valid and parseable JSON object. The function call may optionally be followed by one ; semi-colon. No other content, other than whitespace or valid JavaScript comments, may appear in the JSON-P response, and whitespace and comments must be ignored by the browser JavaScript parser (as would normally be the case).
In order for the browser to be able to know when it should apply such content-filtering to what might otherwise be seen as regular JavaScript content, the MIME-type "application/json-p" and/or "text/json-p" must be declared on the requesting <script> element. Furthermore, the browser can enforce that the response must be of the matching MIME-type, or fail with errors as well.
It is fully known that existing browsers which don't support CORS also will not likely be updated to support this JSON-P strict enforcement, which means that users in those browsers will not be protected by their browser. However, all current browsers can add support for this content filtering, which would provide safer JSON-P for current browser users who are consuming data from web services which do not yet support CORS (or for which the author does not want to use CORS for whatever reason).
It's also recognized that this stricter definition may cause some "JSON-P" transmissions, which rely on the looser interpretation of just arbitrary JavaScript content, to fail. But this could easily be avoided by having the author (and the server) avoid referring to that content with the strictly defined JSON-P MIME-types as described above, which would then prevent the browser from selectively turning on such filtering.
One possible mitigation technique which could help users of older browsers still take advantage of JSON-P safety would be for the author to detect browsers without such support (up for discussion how this feature-test might work) and conditionally route such requests only for those unsafe browsers through a local server-proxy (or even through a client-side flash proxy like flXHR), which could act as a gateway to the content and do the filtering logic described above.
An important side-effect of codifying a standard for JSON-P is that a parser implementation can (and will) be written to check against the strict definition. Such a parser could easily be used in the local server-proxy or the client-side flash proxy to filter unsafe/invalid content that should have been strict JSON-P.
Another interesting use-case for having a standard for JSON-P that a parser can be written to is the notion of being able to unit-test APIs which produce JSON-P output. For instance, such testing is common for JSON APIs, using something like JSONLint, but for JSON-P APIs at the moment, there's no good way to pump the output through some parser to validate it.
This is a first-pass at such a definition for safe JSON-P. I hereby open up the discussion to the community at large to weigh in on the pros and cons and to collaborate together to find a workable definition that can be advocated for to the W3C, WHATWG, and browser vendors.
The best way to get involved is to write blog posts in response to this proposal, and create links to and from such discussions. In addition, this site's markup/code is hosted on github, and you can fork and modify the page to continue the discussion, and then send a pull request to have the site updated. Lastly, you can make short comments in the comment form below, but please keep your comments brief and to the point, as they will be moderated as necessary to keep the discussion on track.
This site is maintained and hosted by Kyle Simpson and Getify Solutions. The site and all its contents are hereby released as public-domain. Stylization choices are deliberately inspired/copied from Douglas Crockford's JSON.org site, given the obvious relationship of the content. Thanks to @hij1nx for the JSON-P logo, obviously also derivative of Douglas Crockford's JSON logo.
"Cowboy" Ben Alman
10/26/2010 4:30pm (CST)
[I]f you're really interested in a safer implementation, you should consider passing the JSON data as a string that must actually be parsed by a JSON parser, and not as an object literal, like so:
specified_function('{"VALID":"JSON STRING"}');Kyle Simpson
10/26/2010 4:44pm (CST)
@Ben -- I think perhaps the security concern wasn't particularly clear. The concern I hear most people cite around JSON-P is that any amount of JavaScript code logic can appear in the response. The goal then is to restrict the response to only being a single function call, that has JSON as the only parameter.
Of course, if the browser were to implement the standard being proposed here, they would not only enforce the single function call part, but also enforce that the parameter part is valid JSON. In that respect, there's no reason the parameter needs to be a string, because before the browser interprets the response into JavaScript, it can simply grab the part between the ( ) and parse it as valid JSON to make sure it passes.
"Cowboy" Ben Alman
10/26/2010 5:23pm (CST)
No matter how a JSON-P "spec" might be defined, as long as it *requires* that a third-party server must execute an arbitrary, overridable JavaScript function in your page, there will be huge security issues, as a malicious JSON-P response could be specifically tailored to exploit a known function in any web site or application, just by invoking it in place of the expected function.
Now, if the JSONP response always utilized the same function name, let's say window.JSONP, then whatever library (or native browser code) is providing that function would be responsible for managing individual JSON-P sub-callbacks and also sanitizing the data. This would ensure that the browser could ensure the response was valid without fear of an unintended malicious attack since all it would send back is:
JSONP({JSON})Of course, while this is far more secure, it's also completely useless, because it doesn't tell the client what JSON response goes with what callback. Still, along those same lines, what about anchoring whatever method that needs to be called to a global JSONP object:
JSONP.req_specific_method({JSON})This would make your proposed browser-based JSON-P response parsing not only easier (since anything not starting with JSONP. could be discarded) but also more secure, since malicious code wouldn't be able to target methods like document.write or postMessage or megaApp.deleteManyItems, since they are not methods of a global JSONP object.
Kyle Simpson
10/26/2010 6:01pm (CST)
@Ben -- It's a valid point that it's possible a malicious service might try to call some other function on the page than the one that the calling logic expected. One possible solution to that is to specify that the browser will enforce that the function being called is the same as the function name in a specific parameter like `callback` (which has become pretty standard across most APIs).
It also wouldn't be that difficult to specify that the function reference can't include `window` or any of the other natives/built-ins (Object, Function, Math, etc). For good measure, `eval` probably needs to be disallowed, too. We really only need to worry about methods which will accept valid JSON (and throwing an error if not), for instance the [ ] array-literal syntax is valid JSON and there are some methods that accept an array. But we'd still only need to worry about methods which would produce some side-effect on the page, not just ones (like JSON.stringify) which might be called but simply just process and return the value. I guess we might worry about calls like `document.write` or `window.open` or `alert`, etc.
I think it's possible to define a black-list of built-ins that should not be allowed.
Again, keep in mind that we're trying to define a "safer" standard for JSON-P, not a completely bullet-proof one. No syntax, even JSON itself, is that "safe".
Peter van der Zee
11/11/2010 3:40pm (CST)
From my proposal for safe JSONP:
I propose a new host object with a very simple api supplied by the browser (hence, host object) that allows you to fetch jsonp (not json) with the same cross domain model as script tags but without sending cookies (maybe only if CORS would allow it). An example is this:
loadJsonp( 'http://example.domain/some/jsonp/source', function onLoad(json){}, function onError(message){} );This new host object (naming is up in the air, I don't care about that) has only one purpose; make the current "jsonp mechanism" safe.
When the function is executed, the browser will fetch the document given by url. It will apply the same cross domain restrictions as it would to a script tag and cookies would not be sent with the request (optional: cookies may be allowed if CORS headers would...). The fetched document should adhere to this production, using ES5 definitions:
When evaluating
JSONPText, theIdentifieris stripped and ignored andJSONTextis ran throughJSON.parse. If parsed successfully theonLoadcallback is called with the object as argument. In any other case (whether it be a bad JSON, bad JSONP or unable to fetch the document) theonErrorfires with some kind of error message.Kyle Simpson
11/15/2010 4:07pm (CST)
A number of people have expressed additional concerns with JSON-P beyond security. Firstly, Peter's proposal seeks in part to gain a better API for requesting and receiving JSON data via JSON-P. Secondly, many people have expressed problems with unreliable error handling around JSON-P responses.
While not a complete solution by any means, I just wanted to point out a possible temporary alternative to address some of those concerns. jXHR (JSON-P XHR) is a small little project that I built some time ago to provide an XHR API "compliant" (mostly) interface to JSON-P request/response handling, in addition to some basic error handling (including timeouts). Future intended features for jXHR include being able to do POSTs with JSON-P (not currently very easy to do) by using a hidden iframe with a form POST, etc.
I agree there are a number of areas where JSON-P needs to improve. This site's main task is to address the security of JSON-P, but the reliability of error handling, API access, flexibility of request method, and other such things are also important for JSON-P's future. As always, if you have any thoughts on such things, please feel free to leave a comment.
Garrett
11/15/2010 10:40pm (CST)
JSON-P can't be aborted reliably. How should a timeout be handled? If the script fails to load after a given amount of time, it might make sense to try again, but how can you do that if you don't know if the script will load?
Kyle Simpson
11/16/2010 7:09am (CST)
@Garret - Those are both valid concerns. I will address both concerns from two perspectives: a) immediate temporary solution b) how this site's effort could possibly improve the "spec" for JSON-P.
As an immediate solution, it's possible (and in fact, I do it) to implement timeouts, and thus aborts/retries, with JSON-P. The key is, as jXHR does, to use a library controlled callback (which in turn calls the user requested callback). If the timeout expires, a flag can be set to tell the library controlled callback to ignore any eventual response it receives. This is effectively an "abort". Moreover, IIRC, in some browsers, if you remove the <script> element from the DOM, the browser will abandon the request (or, perhaps more accurately, ignore any eventual response).
As for how the browser JSON-P "spec" might accomodate things: I'd say that perhaps we could standardize that the way to "abort" a JSON-P request is to in fact do as I just mentioned, which is to remove the element from the DOM. An alternative would be to have some method on the script element, or some property flag, which could be called or set, to indicate to the browser, "I no longer care about this JSON-P response, please ignore it". For timeouts, I don't really see that as something that should be put into the native "spec" behavior, but libraries that do JSON-P can easily implement it if they have a reliable way to abort timed-out requests and perform retries.
I'd say another thing that's important is to improve the reliability of the events (as mentioned earlier in this page). Exactly one of the "error" or "load" events should always fire for every script element. Here's why this is important for JSON-P: it allows a library (like jXHR) to reliably error detect on a JSON-P request.