Support Cloudflare pages/workers

[assaf]

I have a project that's deployed to Cloudflare. The runtime environment behaves like a service worker, so it has fetch but not window, hence getting this error:

ReferenceError: window is not defined
    at node_modules/isomorphic-form-data/lib/browser.js

There's a PR to fix isomorphic-form-data, I doubt it's getting merged: form-data/isomorphic-form-data#6

There's also no XHR (some specific technical reasons for that), so importing masto/fetch also fails:

TypeError: globalThis.XMLHttpRequest is not a constructor
    at node_modules/rollup-plugin-node-polyfills/polyfills/http-lib/capability.js

I wonder if there's an easy fix here, maybe a bundle that doesn't use any polyfills?

[neet]

Please give me some time to resolve this issue as I have no knowledge of Cloudflare pages or Cloud Flare workers. It'd be a good chance to reconsider how polyfills should be handled because Node.js 18 also natively supports FormData.

so importing masto/fetch also fails:

This is unintended. I inadvertently included a module that depends on axios to masto/fetch module too. This will be fixed in another PR #753

[assaf]

Essentially they're running server-side code in V8 as service workers. A service worker can listen and respond to HTTP requests as fetch events: https://developer.mozilla.org/en-US/docs/Web/API/FetchEvent

Service workers don't have access to window, the global context is globalThis, which also exists in Node.

I get the same error using Masto from a Chrome extension, it runs code in the background like a service worker. But Chrome is fine with masto/fetch, I'm guessing the XHR issue is specific to Cloudflare so they throttle outbound requests.

[neet]

XMLHttpRequest

I believe this has been fixed in 4.9.2 #753 though you still have to use masto/fetch. I keep this issue open since /fetch is experimental.

[assaf]

Thanks, the XHR error is gone now.

However, masto/fetch can't upload media. It's using the content-type header without the boundary marker (should look like multipart/form-data; boundary=----WebKitForm…)

[neet]

In masto/fetch, if there is any Blob instance in the parameter, it removes the Content-Type header so fetch automatically sets the boundary, as documented in the spec.

masto.js/src/http/http-native-impl.ts

Lines 35 to 43 in 57c5ebc

	if (
	body instanceof FormData &&
	reqContentType === 'multipart/form-data' &&
	HttpNativeImpl.hasBlob(body)
	) {
	// As multipart form data should contain an arbitrary boundary,
	// leave Content-Type header undefined, so that fetch() API
	// automatically configure Content-Type with an appropriate boundary.
	headers.delete('Content-Type');

Example:

Please check again if you are using the API as shown in the example. If you offer me a minimal repository to reproduce your problem, it would also help resolve your issue.

[assaf]

I'm passing the base-64 encoded string directly since I already have that. This works fine in the browser, fetch can handle data URL strings, not just Blob|File.

Separately, multipart/* MIME types (there's more than one) are supposed to have a boundary, so the server can split requests into distinct parts. Content-Type: multipart/form-data is an incomplete header.

The place where you'd use multipart/form-data on its own is the enctype attribute, which tells the form to use multipart.

[neet]

fetch can handle data URL strings, not just Blob|File.

The place where you'd use multipart/form-data on its own is the enctype attribute,

Ah, I didn't know this. So I should've removed Content-Type regardless it has a Blob or not.

As a workaround, please consider converting the data URI to a Blob by then. I'll keep you informed.

[neet]

This works fine in the browser, fetch can handle data URL strings, not just Blob|File.

@assaf Probably this is because I'm bad at googling, but I could not find any resource that says FormData can handle data URLs. Could you show me an example to reproduce what you meant? Though Content-Type should be left empty anyways

MDN says it's a string as plain text or a Blob instance:

This can be a string or Blob (including subclasses such as File). If none of these are specified the value is converted to a string.
https://developer.mozilla.org/en-US/docs/Web/API/FormData/append#parameters

[assaf]

There are two things going on here.

One is that fetch uses multipart/form-data. Open DevTools in the browser and type this:

const formData = new FormData();
formData.append("image", "");
fetch("/", { method: "POST", body: formData });

You don't have to explicitly set the content type, and setting it to multipart/form-data doesn't do anything useful (that only applies to enctype).

Two, Mastodon specifically handles data: URIs with images. That's not part of any official spec, but some servers know how to parse that:

const image = document.createElement('img');
image.src = '/oops.gif';
image.onload = () => {
  const canvas = document.createElement('canvas');  
  canvas.width = image.width;
  canvas.height = image.height;
  const context = canvas.getContext('2d');
  context.drawImage(image, 0, 0);
  const dataURL = canvas.toDataURL('image/png');
  const formData = new FormData();
  formData.append("image", dataURL);
  fetch("/api/v2/media", { method: "POST", body: formData });
};

(You'll need Authorization header for the above to work)

[neet]

Thank you for explaining. I thought it was about FromData, but now I understand it is Mastodon's undocumented behaviour. I don't know it if is intended, but as you described, setting Content-Type to multipart/form-data doesn't seem to make sense.

I made a PR that removes Content-Type when the body is an instance of FormData regardless if it's containing a Blob #758. It will be released soon

[assaf]

I migrated the Cloudflare project from DIY to masto/fetch, up for 12+ hours, and no issues.

The following work:

• reading data (lookup account, fetch status, search) with and without access token
• iterators (all statuses for account, public timeline) with and without access token
• POST (upload image, publish status)
• PUT (changing attachment description), DELETE (scheduled status)
• uploading images — tested with Blob and base-64 string

[github-actions]

🎉 This issue has been resolved in version 5.0.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀