Testing embedded JavaScript in my headless browser (and a question)

[stroiman]

For my crazy idea, to write a headless browser, I need to have native Go objects exposed to JS code. Tests of JS code generally execute JS code and verify side effect in JS scope.

I now came to the point of implementing an event loop. It did surprise me that v8 doesn't include one (but now I understand why node.js and the browser have different return types for setTimeout and friends).

This is the first test for setTimeout. Apart from verifying the immediatly observed behaviour; there's an implicit verification that no unhandled errors had occurred in JS scope. E.g., the throw new Error() in here causes the test to fail if added; although it has no effect on the actual verification on this particular test; as it's thrown after the observable side effect was created.

The implicit verification occurs in an AfterEach node. Is this a good idea? Or is it an antipattern to do this? And could I place that verification in an DeferCleanup block? The function NewTestContext() creates a v8 context, but also disposes that in a DeferCleanup. It would make a lot of sense to have a variation of this that just implicitly checks for no unhandled errors.

The test is still a bit verbose. As this is the first going down this path, I haven't yet extracted sensible helper functions. The unhandled error check is a perfect case of something that should just be wrapped in a helper.

package scripting_test

import (
	. "github.com/onsi/ginkgo/v2"
	. "github.com/onsi/gomega"
	. "github.com/stroiman/go-dom/browser"
)

var _ = Describe("EventLoop", func() {
	var unhandledErrors []any

	BeforeEach(func() {
		unhandledErrors = nil
	})

	AfterEach(func() {
		Expect(unhandledErrors).To(BeEmpty())
	})

	It("Defers execution", func() {
		ctx := NewTestContext()
		ctx.StartEventLoop()

        // Check for unhandled error.
		ctx.Window().
			AddEventListener("error",
				NewEventHandlerFunc(func(e Event) error {
					unhandledErrors = append(unhandledErrors, e)
					return nil
				}))
		// Set up an event handler to synchronise test code with JS code.
		c := make(chan bool)
		defer close(c)
		ctx.Window().
			AddEventListener("go:home",
				NewEventHandlerFunc(func(e Event) error {
					c <- true
					return nil
				}))

        // Exercise - call setTimeout
		Expect(
			ctx.RunTestScript(`
			  let val; setTimeout(() => {
			    val = 42;
			    window.dispatchEvent(new CustomEvent("go:home"));
			    // throw new Error();
			  }, 1); 
			  val`,
			),
		).To(BeNil())

        // Wait for the callback to have been executed
		<-c
        // Verify observable side effect from the callback
		Expect(ctx.RunTestScript(`val`)).To(BeEquivalentTo(42))
	})
})

[onsi]

Hey there, cool project! Yes this pattern makes sense to me and mirrors stuff I've done in the past in similar situations. It's also how the gleak gomega library works to assert there were no leaking goroutines.

And yes, you can use DeferCleanup instead. In fact, I'd recommend doing so and. Looking ahead a bit, I'd push the error checking into TestContext by default and have the NewTestContext constructor add the DeferCleanup. For tests that need to make assertions against errors you can either add a new constructor or make TestContext smarter and give yourself the ability to query the errors under test.

Assuming TestContext does the DeferCleanup there are a few additional tweaks that I'd recommend.

Here's a version that uses a few more of Gomega's affordances to make to guard against the test locking up on a chanel read that never happens:

It("Defers execution", func() {
	ctx := NewTestContext()
	ctx.StartEventLoop()

	c := make(chan any)
	ctx.Window().
		AddEventListener("go:home",
			NewEventHandlerFunc(func(e Event) error {
				close(c) // just close the channel to signify the event was received
				return nil
			}))

	// Exercise - call setTimeout
	Expect(
		ctx.RunTestScript(`
			let val; setTimeout(() => {
			val = 42;
			window.dispatchEvent(new CustomEvent("go:home"));
			// throw new Error();
			}, 1); 
			val`,
		),
	).To(BeNil())

	Eventually(c).Should(BeClosed()) //wrap this in an eventually so your test doesn't block.  This will poll the channel up to the default timeout (of 1 second)
	Expect(ctx.RunTestScript(`val`)).To(BeEquivalentTo(42))
})

But my sense is you are only using the go:home event so you can synchronize the test with the setTimeout - not because you are actually interested in testing the event itself.

If so, I'd leverage the fact that Javascript runtimes are single-threaded and do this instead:

It("Defers execution", func() {
	ctx := NewTestContext()
	ctx.StartEventLoop()

	// Exercise - call setTimeout
	Expect(
		ctx.RunTestScript(`
			let val; setTimeout(() => {
			val = 42;
			// throw new Error();
			}, 1); 
			val`,
		),
	).To(BeNil())

	Eventually(ctx.RunTestScript).WithArguments("val").Should(BeEquivalentTo(42))
})

This gets rid of the incidental stuff. Here Eventually will poll ctx.RunTestScript, passing in "val" as an argument. It will continue polling for up to the default timeout (1 second) until the value is 42. If it doesn't get there in time, the test will fail with a timeout error. You can learn more about Eventually and Consistently here and I would recommend embracing them as they will make testing this sort of codebase a lot more fluent. This sort of approach works in this context because JS is single-threaded and there is no race condition between reading val and writing val.

Lastly, if you're interested - I built a thing called biloba to do headless browser testing in Ginkgo and Gomega. The project itself might be interesting (obviously, different goals than what you're building) but the test suite might be a good resource as it is in a similar domain as what you are working on.

[stroiman]

Hey, thanks a lot for the feedback, greatly appreciated.

I actually had a good feeling about the approach and I have already moved a lot to the NewTestContext; which now by default checks for unhandled errors in a DeferCleanup, but that can be disabled using an option, which is used right now in the tests verifying that JS code can listen to that event.

But my sense is you are only using the go:home event so you can synchronize the test with the setTimeout - not because you are actually interested in testing the event itself.

You are absolutely right, it's only for test synchronization. I generally prefer to avoid polling as much as possible, so the test will proceed as soon as it's ready. But the version you showed is certainly less code :)

But eventually, I'll probably move that synchronisation to the event loop itself, with functions like RunAll, that executes until there are no more enqueued work, or FastForward that simulates time passes, and handles all timeouts/intervals that has been registered in the meantime, allowing tests of throttling behaviour to not have to wait for the throttle timeout to occur. (lolex, which is a JS time mocking framework have these capabilities)

But you did show some things I wasn't aware of, e.g. the WithArguments (which is really nice for that specific scenario) and Eventually(...).Should(BeClosed()) - didn't know about that either.

[onsi]

Makes sense all around. FWIW the polling is quite frequent and generally hasn't been a source of significant slow-down in my experience. But since you're building the event loop yourself, implementing RunAll/FastForward etc. makes more sense and will give you much more precision and zero flakiness.

In case you want to guard against tests that hang forever, here's a synchronous non-polling approach that uses Ginkgo's SpecContext to handle timeouts:

It("Defers execution", SpecTimeout(time.Second), func(spCtx SpecContext) {
	ctx := NewTestContext()
	ctx.StartEventLoop()

	c := make(chan any)
	ctx.Window().
		AddEventListener("go:home",
			NewEventHandlerFunc(func(e Event) error {
				close(c) // just close the channel to signify the event was received
				return nil
			}))

	// Exercise - call setTimeout
	Expect(
		ctx.RunTestScript(`
			let val; setTimeout(() => {
			val = 42;
			window.dispatchEvent(new CustomEvent("go:home"));
			// throw new Error();
			}, 1); 
			val`,
		),
	).To(BeNil())

	select {
	case <-c:
		Expect(ctx.RunTestScript(`val`)).To(BeEquivalentTo(42))
	case <-spCtx.Done():
		return
	}
})

(just sharing to point you at some of the other toys and tools at your disposal)

[stroiman]

Thanks for that suggestion also. I didn't know about the Done() part either.