Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support for BTF_KIND_DECL_TAG and BTF_KIND_TYPE_TAG #713

Open
1 of 3 tasks
lmb opened this issue Jun 15, 2022 · 4 comments
Open
1 of 3 tasks

Support for BTF_KIND_DECL_TAG and BTF_KIND_TYPE_TAG #713

lmb opened this issue Jun 15, 2022 · 4 comments
Labels
enhancement New feature or request

Comments

@lmb
Copy link
Collaborator

lmb commented Jun 15, 2022 •
edited
Loading

Linux 5.18 added a two new BTF kinds DECL_TAG and TYPE_TAG. TYPE_TAG seems fine, the encoding for DECL_TAG looks like a headache.

Most relevant Linux commit: torvalds/linux@7472d5a

pahole only generates these kinds from v1.23 onwards:

clang-14 is also required: https://reviews.llvm.org/D111199

  • Parse / skip the new types
  • Omit encoding new types in Spec.marshal if the kernel doesn't understand them
  • Export types from btf
@lmb lmb added the bug Something isn't working label Jun 15, 2022
lmb added a commit to ti-mo/ebpf that referenced this issue Jun 15, 2022
@lmb
libbpf-compat: skip tests that fail on 5.18
f4ed59b 
5.18 added two new BTF kinds TYPE_TAG and DECL_TAG which we
don't support at the moment.

See cilium#713
lmb added a commit to ti-mo/ebpf that referenced this issue Jun 15, 2022
@lmb
libbpf-compat: skip tests that fail on 5.18
56a0334 
5.18 added two new BTF kinds TYPE_TAG and DECL_TAG which we
don't support at the moment.

See cilium#713
@ti-mo
Copy link
Collaborator

ti-mo commented Jun 28, 2022

With BTF-enabled kernels becoming generally available in distro's, we might need to think about tolerating/ignoring unknown types so older tools written with CO-RE are forwards-compatible with newer kernels.

Time for a btf.ReaderOptions of some sort? :)

@lmb
Copy link
Collaborator Author

lmb commented Jun 29, 2022

BTF isn't self describing unfortunately, so skipping unknown types is not possible.

@lmb lmb self-assigned this Jul 7, 2022
lmb added a commit to cilium/ci-kernels that referenced this issue Jul 19, 2022
@lmb
build pahole v1.23 from source
95cdf4d 
We need pahole >= 1.23 so it generates BTF_KIND_DECL_TAG and BTF_KIND_TYPE_TAG.

See cilium/ebpf#713
lmb added a commit to ti-mo/ebpf that referenced this issue Jul 19, 2022
@lmb
libbpf-compat: skip tests that fail on 5.18
929b669 
5.18 added two new BTF kinds TYPE_TAG and DECL_TAG which we
don't support at the moment.

See cilium#713
@lmb lmb mentioned this issue Jul 19, 2022
Merged
lmb added a commit to cilium/ci-kernels that referenced this issue Jul 19, 2022
@lmb
build pahole v1.23 from source
1487523 
We need pahole >= 1.23 so it generates BTF_KIND_DECL_TAG and BTF_KIND_TYPE_TAG.

See cilium/ebpf#713
@lmb
Copy link
Collaborator Author

lmb commented Jul 20, 2022

My current reading of this is that vmlinux BTF will only contain these new types if:

  • clang >= 14
  • pahole >= 1.23
  • the kernel is built with clang instead of gcc

Not sure how many production kernels use clang instead of gcc.

lmb added a commit to ti-mo/ebpf that referenced this issue Jul 20, 2022
@lmb
libbpf-compat: skip tests that fail on 5.18
5a756ba 
5.18 added two new BTF kinds TYPE_TAG and DECL_TAG which we
don't support at the moment.

See cilium#713
ti-mo pushed a commit that referenced this issue Jul 20, 2022
@lmb @ti-mo
libbpf-compat: skip tests that fail on 5.18
dd196e5 
5.18 added two new BTF kinds TYPE_TAG and DECL_TAG which we
don't support at the moment.

See #713
lmb added a commit to lmb/ebpf that referenced this issue Jul 25, 2022
@lmb
btf: add limited support for BTF_KIND_DECL_TAG and BTF_KIND_TYPE_TAG
bc34e86 
Parse decl and type tags from BTF so that we can read BTF on new kernels.
The new types are not exported since the encoding of declTag.Index especially
is cumbersome to use. We can export the types later if necessary.

Treat typeTag as a qualifier, since accoriding to the BTF documentation it
is usually used in a chain of types:

    ptr -> [type_tag]*
        -> [const | volatile | restrict | typedef]*
        -> base_type

Updates cilium#713
lmb added a commit to lmb/ebpf that referenced this issue Jul 25, 2022
@lmb
btf: add limited support for BTF_KIND_DECL_TAG and BTF_KIND_TYPE_TAG
110e43a 
Parse decl and type tags from BTF so that we can read BTF on new kernels.
The new types are not exported since the encoding of declTag.Index especially
is cumbersome to use. We can export the types later if necessary.

Treat typeTag as a qualifier, since accoriding to the BTF documentation it
is usually used in a chain of types:

    ptr -> [type_tag]*
        -> [const | volatile | restrict | typedef]*
        -> base_type

Updates cilium#713
lmb added a commit to lmb/ebpf that referenced this issue Jul 26, 2022
@lmb
btf: add limited support for BTF_KIND_DECL_TAG and BTF_KIND_TYPE_TAG
e10a92a 
Parse decl and type tags from BTF so that we can read BTF on new kernels.
The new types are not exported since the encoding of declTag.Index especially
is cumbersome to use. We can export the types later if necessary.

Treat typeTag as a qualifier, since accoriding to the BTF documentation it
is usually used in a chain of types:

    ptr -> [type_tag]*
        -> [const | volatile | restrict | typedef]*
        -> base_type

Updates cilium#713
lmb added a commit to lmb/ebpf that referenced this issue Aug 22, 2022
@lmb
btf: add limited support for BTF_KIND_DECL_TAG and BTF_KIND_TYPE_TAG
14c9289 
Parse decl and type tags from BTF so that we can read BTF on new kernels.
The new types are not exported since the encoding of declTag.Index especially
is cumbersome to use. We can export the types later if necessary.

Treat typeTag as a qualifier, since accoriding to the BTF documentation it
is usually used in a chain of types:

    ptr -> [type_tag]*
        -> [const | volatile | restrict | typedef]*
        -> base_type

Updates cilium#713
lmb added a commit that referenced this issue Aug 23, 2022
@lmb
btf: add limited support for BTF_KIND_DECL_TAG and BTF_KIND_TYPE_TAG
89c61ac 
Parse decl and type tags from BTF so that we can read BTF on new kernels.
The new types are not exported since the encoding of declTag.Index especially
is cumbersome to use. We can export the types later if necessary.

Treat typeTag as a qualifier, since accoriding to the BTF documentation it
is usually used in a chain of types:

    ptr -> [type_tag]*
        -> [const | volatile | restrict | typedef]*
        -> base_type

Updates #713
@lmb lmb added enhancement New feature or request and removed bug Something isn't working labels Oct 27, 2022
@lmb
Copy link
Collaborator Author

lmb commented Jan 12, 2023

An idea I've been kicking around: DECL_TAG and TYPE_TAG are a bit of a bodge to add stringly typed metadata to the BTF wire format. What if we didn't represent them as Type, but instead added the metadata to Pointer or whatever?

type Pointer struct {
	...
	Tags []string
}

When decoding BTF we'd add TYPE_TAGs to Pointer.Tags and during marshaling we'd create TYPE_TAG on the fly. Similar for DECL_TAG which applies to Struct and Union (and others?) This is the best solution from an API perspective, it's very easy to add and remove tags from a Type.

Downsides:

  • We're creating a mismatch with upstream BTF. With this approach it would be impossible to get the type ID of a *_TAG, so if we ever need to pass such an ID to the kernel we'd be in a pickle.
  • We might also have to add Tags to a large number of Types.
  • We'd have to follow tag2 -> tag1 -> type chains during decoding somehow.

@lmb lmb mentioned this issue Jan 12, 2023
Open
@lmb lmb removed their assignment Mar 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lmb @ti-mo