HTTP imports on node

node v18 shall have ESM import support over HTTPS remotely and HTTP locally and is currently under a flag.

This is experimental and things can change drastically, some of the feature might be removed as well, so take it with a pinch of salt.

Here is an example:

_Code sample:

import helloWorld from "https://h3manth.com/hello-world.mjs";

console.log(helloWorld()); // Log Hello World

import helloWorld from "http://localhost:1337/hello-world.mjs";

console.log(helloWorld()); // Log Hello World

As you might have noticed, this is still under an experimental flag: --experimental-network-imports

For now, the --experimental-network-imports flag is required to enable loading
resources over http: or https:. In the future, a different mechanism will be
used to enforce this. Opt-in is required to prevent transitive dependencies
inadvertently using potentially mutable state that could affect reliability
of Node.js applications.

The contents of https://h3manth.com/hello-world.mjs:

export default function helloWorld () {
  console.log('Hello, World!')
}

The most important thing to notice here is the MIME type of the URL.

$ curl -s -o /dev/null -w '%{content_type}' 'https://h3manth.com/hello-world.mjs'

application/javascript

The hosted mjs should respond with application/javascript as it's content-type or else it will fail to load.

Authorization, Cookie, and Proxy-Authorization headers are not sent to the server and CORS policies/headers are neither sent nor enforced.

Some of the common errors:

• ERR_NETWORK_IMPORT_BAD_RESPONSE occures when the URL gives a bad response, like 404

import helloWorld from "https://h3manth.com/foo";

Error [ERR_NETWORK_IMPORT_BAD_RESPONSE]: import 'https://h3manth.com/foo' received a bad response: HTTP response returned status code of 404

• ERR_NETWORK_IMPORT_DISALLOWED will occur when the request cannot redirect to non-network location, if HTTP is used for non-local resources.

import helloWorld from "https://h3manth.com/foo.mjs"; // Say this redirects to ftp://h3manth.com/bar.mjs

Error [ERR_NETWORK_IMPORT_DISALLOWED]: import of 'ftp://h3manth.com/bar.mjs' by 'https://h3manth.com/foo.mjs' is not support: cannot redirect to non-network location

• ERR_UNKNOWN_MODULE_FORMAT occures when the URL is not an ESM.

import helloWorld from "https://h3manth.com/hello.js";

RangeError [ERR_UNKNOWN_MODULE_FORMAT]: Unknown module format: null for URL https://h3manth.com/hello.js

• ERR_UNSUPPORTED_ESM_URL_SCHEME if your URL scheme is not supported by the default ESM loader

import helloWorld from "ftp://foo.js";

console.log(helloWorld())

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, https, http are supported by the default ESM loader. Received protocol 'ftp:'

How do the fetch works?

fetchModule does the basic sanity checks and also checks if it is already present in cache (cacheForGET) if not it invokes fetchWithRedirects

function fetchModule(parsed, { parentURL }) {
  const { href } = parsed;
  const existing = cacheForGET.get(href);
  if (existing) {
    return existing;
  }
  if (parsed.protocol === 'http:') {
    return PromisePrototypeThen(isLocalAddress(parsed.hostname), (is) => {
      // Makes few checks for ERR_NETWORK_IMPORT_DISALLOWED
      return fetchWithRedirects(parsed);
    });
  }
  return fetchWithRedirects(parsed);
}

function fetchWithRedirects(parsed) {
  const existing = cacheForGET.get(parsed.href);
  if (existing) {
    return existing;
  }
  const handler = parsed.protocol === 'http:' ? HTTPGet : HTTPSGet;
  const result = new Promise((fulfill, reject) => {
    const req = handler(parsed, {
      headers: {
        Accept: '*/*'
      }
    })
    .on('error', reject)
    .on('response', (res) => {
      // Checks for multiple error cases that were mentoined above
      // Caches the content if required 
      // Returns a promise which shall resolve to the module's content
    }
}

let HTTPSAgent;
function HTTPSGet(url, opts) {
  const https = require('https'); // For HTTPGet, we use the `http` builtin
  HTTPSAgent ??= new https.Agent({
    keepAlive: true
  });
  return https.get(url, {
    agent: HTTPSAgent,
    ...opts
  });
}

So, summerizing the status of this experimental feature:

• Disabled by default - requires --experimental-https-modules flag
• Redirects - act like web
• Selective HTTP support - only works on loopback
• Debugging HTTPS on a local machine is painful - use loopback + http:
• Cannot access node: builtins - banned anything except http: and https: deps.

Source of truth is the commit on the main branch of node src.

Feel free to share this article. You may as well ping me on Twitter.

Published 11 Feb 2022