Using wamr in HarfBuzz

[behdad]

Hi,

Last year I added the ability to HarfBuzz to run wasm from a font file to do text shaping. In doing so I hit a few shortcomings in the wamr runtime, which I like to discuss here:

• Stable API is highly desired. I just updated my checkout and noticed a change in the wasm_runtime_lookup_function() function signature.

• Multi-threaded: HarfBuzz is a multi-thread-enabled library. I see in:
  Can I use wamr in multithreaded environment? #3012
  that multi-threading seems to be supported. Though since HarfBuzz doesn't spawn and bring down the threads, we need to do some tricks to call the respective functions. I hope their overhead is not high? Also, last I checked the code, it was using some global state variables. Have they been removed? Or just made into TLS?

• Re-enterant: Likewise, we need to be able to run wasm from multiple objects in the same thread. This suggests that it might be possible:
  Is it safe to have two `exec_env`s at the same time? #2381

• wasm_runtime_full_init(). Not clear how to know whether to call it or now. Basically the global state makes it rather impossible to use wamr from a library where other libraries or the user might also be using it. Unless we keep a private copy of wamr, which is not desirable.

• obj2ref / ref2obj are not type-safe. I have implemented type-safety for my own types using them. But if the user also uses that API, I'm doomed. Basically the code assumes that there's only one class of types that are passed back and force between the runtime and the host.

• Work-timer. Limiting how much work the untrusted wasm can do, and interrupt it, is of paramount interest to us. Maybe it will never get implemented?

• Dynamic memory growth.

• Multiple memories will be very interesting to us.

That's about it for now. Thanks for reading.

[wenyongh]

Hi, thanks for using WAMR in HarfBuzz! We are glad to support it and make WAMR used in more and more projects.

• Stable API is highly desired. I just updated my checkout and noticed a change in the wasm_runtime_lookup_function() function signature.

Yes, the unused signature argument in wasm_runtime_lookup_function has existed before (and also in) WAMR-1.3.2, to keep the API compatibility, it is unchanged for a very long time. But recently the GC feature was merged and we are implementing the memory64 feature, which require AOT ABI and API changes, like AOT file format, AOT module/memory instance layout, supporting 64-bit linear memory offset, so we refactored the API and data structures to better meet the requirements, see #3116, #3202, #3091.

• Multi-threaded: HarfBuzz is a multi-thread-enabled library. I see in:
  Can I use wamr in multithreaded environment? #3012
  that multi-threading seems to be supported. Though since HarfBuzz doesn't spawn and bring down the threads, we need to do some tricks to call the respective functions. I hope their overhead is not high? Also, last I checked the code, it was using some global state variables. Have they been removed? Or just made into TLS?

Do you mean HarfBuzz doesn't use pthread_create like API to create thread, and you had to use wasm_runtime_spawn_thread or wasm_runtime_spawn_exec_env? I think the overhead should be small, please refer to https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/core/iwasm/libraries/thread-mgr/thread_manager.c#L493-L584, it is mainly to create some necessary resources.
For the global state variables, which variables do you refer to?

• Re-enterant: Likewise, we need to be able to run wasm from multiple objects in the same thread. This suggests that it might be possible:
  Is it safe to have two exec_envs at the same time? #2381
• wasm_runtime_full_init(). Not clear how to know whether to call it or now. Basically the global state makes it rather impossible to use wamr from a library where other libraries or the user might also be using it. Unless we keep a private copy of wamr, which is not desirable.

Not sure what libraries are HarfBuzz using? If there are two libraries using WAMR simultaneously, maybe we can add a reference count, and initialize runtime only when ref count is 0, somewhat like ref_count in shared memory: https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/core/iwasm/common/wasm_shared_memory.c#L77-L108.

• obj2ref / ref2obj are not type-safe. I have implemented type-safety for my own types using them. But if the user also uses that API, I'm doomed. Basically the code assumes that there's only one class of types that are passed back and force between the runtime and the host.

Do you apply patch to WAMR? It seems that reference types are rarely used now.

• Work-timer. Limiting how much work the untrusted wasm can do, and interrupt it, is of paramount interest to us. Maybe it will never get implemented?

What the untrusted wasm means? Do you mean to implement some mechanism like watchdog?

• Dynamic memory growth.
• Multiple memories will be very interesting to us.

It is under discussion and hope we can start to implement multi-memory after memory64 feature is implement.

[behdad]

Thanks for WAMR and thanks for getting back to me.

• Multi-threaded: HarfBuzz is a multi-thread-enabled library. I see in:
  Can I use wamr in multithreaded environment? #3012
  that multi-threading seems to be supported. Though since HarfBuzz doesn't spawn and bring down the threads, we need to do some tricks to call the respective functions. I hope their overhead is not high? Also, last I checked the code, it was using some global state variables. Have they been removed? Or just made into TLS?

Do you mean HarfBuzz doesn't use pthread_create like API to create thread,

Yes. HarfBuzz doesn't deal with threads directly. It's just that our objects are safe to use from multiple threads simultaneously. We like to extend that to our wasm runner as well.

and you had to use wasm_runtime_spawn_thread or wasm_runtime_spawn_exec_env?

I haven't studied those yet. I will and report back.

I think the overhead should be small, please refer to https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/core/iwasm/libraries/thread-mgr/thread_manager.c#L493-L584, it is mainly to create some necessary resources.

Thanks. Will do.

For the global state variables, which variables do you refer to?

The very fact that wasm_runtime_full_init() takes input that is then used globally, if I understand correctly. Different clients might want to set it up differently. Basically a context object would take care of that.

• Re-enterant: Likewise, we need to be able to run wasm from multiple objects in the same thread. This suggests that it might be possible:
  Is it safe to have two exec_envs at the same time? #2381
• wasm_runtime_full_init(). Not clear how to know whether to call it or now. Basically the global state makes it rather impossible to use wamr from a library where other libraries or the user might also be using it. Unless we keep a private copy of wamr, which is not desirable.

Not sure what libraries are HarfBuzz using? If there are two libraries using WAMR simultaneously, maybe we can add a reference count, and initialize runtime only when ref count is 0, somewhat like ref_count in shared memory: https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/core/iwasm/common/wasm_shared_memory.c#L77-L108.

HarfBuzz itself doesn't use any libraries. But the clienst might do use other WAMR clients, or use WAMR themselves. See above. Refcount would have worked if there weren't any setup parameters...

• obj2ref / ref2obj are not type-safe. I have implemented type-safety for my own types using them. But if the user also uses that API, I'm doomed. Basically the code assumes that there's only one class of types that are passed back and force between the runtime and the host.

Do you apply patch to WAMR? It seems that reference types are rarely used now.

No I don't apply any patches to WAMR. Reference type are indeed rarely used. But obj2ref / ref2obj work with other types as well, no? This is what I have in HarfBuzz:

#define HB_REF2OBJ(obj) \
  hb_##obj##_t *obj = nullptr; \
  HB_STMT_START { \
    (void) wasm_externref_ref2obj (obj##ptr, (void **) &obj); \
    /* Check object type. */ \
    /* This works because all our objects have the same hb_object_t layout. */ \
    if (unlikely (hb_##obj##_get_user_data (obj, &_hb_wasm_ref_type_key) != \
		  (void *) hb_wasm_ref_type_##obj)) \
      obj = hb_##obj##_get_empty (); \
  } HB_STMT_END

#define HB_OBJ2REF(obj) \
  uint32_t obj##ref = nullref; \
  HB_STMT_START { \
    hb_##obj##_set_user_data (obj, &_hb_wasm_ref_type_key, \
			      (void *) hb_wasm_ref_type_##obj, \
			      nullptr, false); \
    (void) wasm_externref_obj2ref (module_inst, obj, &obj##ref); \
  } HB_STMT_END

Basically, if another library / client passes random object to HarfBuzz, we can crash. And that's not desirable.

• Work-timer. Limiting how much work the untrusted wasm can do, and interrupt it, is of paramount interest to us. Maybe it will never get implemented?

What the untrusted wasm means? Do you mean to implement some mechanism like watchdog?

Yeah watchdog. Imagine a font is downloaded by your browser from the web, and passed to HarfBuzz. If the font has the Wasm table, we pass that code to WAMR to run. That's what I mean by untrusted. We don't want that to stall or crash the browser.

• Dynamic memory growth.
• Multiple memories will be very interesting to us.

It is under discussion and hope we can start to implement multi-memory after memory64 feature is implement.

Thanks.

[wenyongh]

Do you mean HarfBuzz doesn't use pthread_create like API to create thread,

Yes. HarfBuzz doesn't deal with threads directly. It's just that our objects are safe to use from multiple threads simultaneously. We like to extend that to our wasm runner as well.

and you had to use wasm_runtime_spawn_thread or wasm_runtime_spawn_exec_env?

I haven't studied those yet. I will and report back.

OK, maybe you can refer to https://github.com/bytecodealliance/wasm-micro-runtime/tree/main/samples/spawn-thread, https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/doc/pthread_library.md, and https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/core/iwasm/include/wasm_export.h#L1362.

I think the overhead should be small, please refer to https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/core/iwasm/libraries/thread-mgr/thread_manager.c#L493-L584, it is mainly to create some necessary resources.

Thanks. Will do.

For the global state variables, which variables do you refer to?

The very fact that wasm_runtime_full_init() takes input that is then used globally, if I understand correctly. Different clients might want to set it up differently. Basically a context object would take care of that.

WAMR is designed to be well deployed in many devices, and some devices may haven't heap allocator (no malloc like function) and thread local storage, so wasm_runtime_full_init may not use thread local object, and may not create a context object first with malloc function - it is supposed that runtime can initialize a heap allocator from a global buffer array passed by the caller and then allocate memory from it.

Not sure what libraries are HarfBuzz using? If there are two libraries using WAMR simultaneously, maybe we can add a reference count, and initialize runtime only when ref count is 0, somewhat like ref_count in shared memory: https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/core/iwasm/common/wasm_shared_memory.c#L77-L108.

HarfBuzz itself doesn't use any libraries. But the clienst might do use other WAMR clients, or use WAMR themselves. See above. Refcount would have worked if there weren't any setup parameters...

Yes, we will have an internal discussion and check whether to add ref count for wasm_runtime_full_init.

No I don't apply any patches to WAMR. Reference type are indeed rarely used. But obj2ref / ref2obj work with other types as well, no? This is what I have in HarfBuzz:

#define HB_REF2OBJ(obj) \
  hb_##obj##_t *obj = nullptr; \
  HB_STMT_START { \
    (void) wasm_externref_ref2obj (obj##ptr, (void **) &obj); \
    /* Check object type. */ \
    /* This works because all our objects have the same hb_object_t layout. */ \
    if (unlikely (hb_##obj##_get_user_data (obj, &_hb_wasm_ref_type_key) != \
		  (void *) hb_wasm_ref_type_##obj)) \
      obj = hb_##obj##_get_empty (); \
  } HB_STMT_END

#define HB_OBJ2REF(obj) \
  uint32_t obj##ref = nullref; \
  HB_STMT_START { \
    hb_##obj##_set_user_data (obj, &_hb_wasm_ref_type_key, \
			      (void *) hb_wasm_ref_type_##obj, \
			      nullptr, false); \
    (void) wasm_externref_obj2ref (module_inst, obj, &obj##ref); \
  } HB_STMT_END

Basically, if another library / client passes random object to HarfBuzz, we can crash. And that's not desirable.

Do you access the host object inside runtime? Or just in the native API? It is supposed that the host object is passed from host native to runtime and then passed back to native API, and runtime doesn't know its layout and should not access it. What the native API gets should be the object that host native passes to the wasm function.

• Work-timer. Limiting how much work the untrusted wasm can do, and interrupt it, is of paramount interest to us. Maybe it will never get implemented?

What the untrusted wasm means? Do you mean to implement some mechanism like watchdog?

Yeah watchdog. Imagine a font is downloaded by your browser from the web, and passed to HarfBuzz. If the font has the Wasm table, we pass that code to WAMR to run. That's what I mean by untrusted. We don't want that to stall or crash the browser.

Maybe you can add the watchdog in another thread by yourself, and terminate the wasm instance if needed by calling wasm_runtime_terminate? Refer to https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/core/iwasm/include/wasm_export.h#L953-L969. The wasm instance will throw "terminated by user" exception and terminate the function execution.

[wenyongh]

#3263

[behdad]

Thank you. I also don't mind working on some of these. I'll come back when I have something to show.