Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: replace github.com/ghodss/yaml with gopkg.in/yaml.v3 #230

Merged
merged 2 commits into from
Mar 11, 2023
Merged

refactor: replace github.com/ghodss/yaml with gopkg.in/yaml.v3 #230

merged 2 commits into from
Mar 11, 2023

Conversation

Juneezee
Copy link
Contributor

@Juneezee Juneezee commented Mar 10, 2023

The package github.com/ghodss/yaml is no longer actively maintained. See discussion in ghodss/yaml#75 and ghodss/yaml#80. sigs.k8s.io/yaml is a permanent fork of github.com/ghodss/yaml.

The notable change is that github.com/ghodss/yaml uses gopkg.in/yaml.v2 v2.2.2, but sigs.k8s.io/yaml uses gopkg.in/yaml.v2 v2.4.0. Changes can be seen here v2.2.2...v2.4.0, mostly bug fixes.

@cole-miller
Copy link
Contributor

cole-miller commented Mar 10, 2023

Thanks for the PR and for bringing up the maintenance status of github.com/ghodss/yaml. Since our YAML needs are so simple, perhaps we could get away with just using gopkg.in/yaml.v2 (or v3) directly and eliminating a dependency?

@cole-miller cole-miller self-assigned this Mar 10, 2023
@tomponline
Copy link
Member

Thanks for the PR and for bringing up the maintenance status of github.com/ghodss/yaml. Since our YAML needs are so simple, perhaps we could get away with just using gopkg.in/yaml.v2 (or v3) directly and eliminating a dependency?

That would be preferable.

@cole-miller
Copy link
Contributor

@Juneezee, would you be up for reworking this PR to use gopkg.in/yaml.v3?

At the time of making this commit, the package `github.com/ghodss/yaml`
is no longer actively maintained.

Signed-off-by: Eng Zer Jun <[email protected]>
@Juneezee Juneezee changed the title refactor: replace github.com/ghodss/yaml with sigs.k8s.io/yaml refactor: replace github.com/ghodss/yaml with gopkg.in/yaml.v3 Mar 11, 2023
@Juneezee
Copy link
Contributor Author

@Juneezee, would you be up for reworking this PR to use gopkg.in/yaml.v3?

Done!

Copy link
Contributor

@cole-miller cole-miller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just ran some tests locally and it seems that -- unlike github.com/ghodss/yaml -- gopkg.in/yaml.v3 does some case conversion by default:

package main

import (
	orig "github.com/ghodss/yaml"
	yamlv3 "gopkg.in/yaml.v3"
)

type NodeInfo struct {
	ID uint64
	Address string
	Role int
}

func main() {
	info := NodeInfo{ 5, "abc", 7 }
	data, _ := orig.Marshal(&info)
	println(string(data))
	data, _ = yamlv3.Marshal(&info)
	println(string(data))
}
$ go run
Address: abc
ID: 5
Role: 7

id: 5
address: abc
role: 7

Could you add yaml: annotations on this declaration so that the replacement will be backwards-compatible? (We want each new version of go-dqlite to be able to read node store and info files written by a previous version.)

type NodeInfo struct {
ID uint64
Address string
Role NodeRole
}

@Juneezee
Copy link
Contributor Author

I just ran some tests locally and it seems that -- unlike github.com/ghodss/yaml -- gopkg.in/yaml.v3 does some case conversion by default

Yup, I noticed the same too. The tests are failing with both gopkg.in/yaml.v2 and gopkg.in/yaml.v3 (https://github.com/Juneezee/go-dqlite/actions/runs/4390622459).

gopkg.in/yaml.v2 and gopkg.in/yaml.v3 marshal and unmarshal using the field name lowercased as default key if we don't explicitly specify a yaml field tag. (https://pkg.go.dev/gopkg.in/yaml.v3#Marshal)

Unlike `github.com/ghodss/yaml`, `gopkg.in/yaml.v3` marshals using the
field name lowercased as the default key [1].

[1]: https://pkg.go.dev/gopkg.in/yaml.v3#Marshal
Signed-off-by: Eng Zer Jun <[email protected]>
Copy link
Contributor

@cole-miller cole-miller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@cole-miller cole-miller merged commit 394129c into canonical:master Mar 11, 2023
@MathieuBordere
Copy link

MathieuBordere commented Mar 13, 2023

seems to hit a segfault on s390x https://launchpadlibrarian.net/655817390/buildlog_ubuntu-focal-s390x.go-dqlite_1.11.7+git5-g394129c~focal1_BUILDING.txt.gz

--- FAIL: TestNew_JoinerRestart (0.28s)
    app_test.go:1073: 19:04:03.209 - 4: DEBUG: new connection from 127.0.0.1:38408
    app_test.go:1073: 19:04:03.212 - 4: DEBUG: attempt 1: server 127.0.0.1:9001: connected
    app_test.go:1073: 19:04:03.213 - 4: DEBUG: new connection from 127.0.0.1:38410
    app_test.go:1073: 19:04:03.306 - 4: DEBUG: new connection from 127.0.0.1:38426
    app_test.go:1073: 19:04:03.309 - 5: DEBUG: attempt 1: server 127.0.0.1:9001: connected
    app_test.go:1073: 19:04:03.312 - 5: DEBUG: new connection from 127.0.0.1:33438
    app_test.go:1073: 19:04:03.312 - 4: DEBUG: new connection from 127.0.0.1:38442
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x12f8e42]

goroutine 58 [running]:
testing.tRunner.func1(0xc0003fc000)
	/usr/lib/go-1.13/src/testing/testing.go:874 +0x41a
panic(0x154fb00, 0x1949de0)
	/usr/lib/go-1.13/src/runtime/panic.go:679 +0x1d4
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.handleErr(0xc00011d9c8)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/yaml.go:294 +0xb2
panic(0x154fb00, 0x1949de0)
	/usr/lib/go-1.13/src/runtime/panic.go:679 +0x1d4
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.yaml_parser_parse_node(0xc000166000, 0xc0001662b0, 0x100000000000000, 0x0)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/parserc.go:564 +0x4c2
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.yaml_parser_state_machine(0xc000166000, 0xc0001662b0, 0xc0000ce280)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/parserc.go:170 +0x1c6
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.yaml_parser_parse(0xc000166000, 0xc0001662b0, 0x3ff83337460)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/parserc.go:129 +0x8a
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.(*parser).peek(0xc000166000, 0x0)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/decode.go:106 +0x42
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.(*parser).parse(0xc000166000, 0x0)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/decode.go:149 +0x38
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.(*parser).parseChild(0xc000166000, 0xc0000ce280, 0x0)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/decode.go:197 +0x2a
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.(*parser).document(0xc000166000, 0x300000000000030)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/decode.go:206 +0xa8
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.(*parser).parse(0xc000166000, 0x0)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/decode.go:159 +0x154
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.unmarshal(0xc00022e480, 0x38, 0x238, 0x1512620, 0xc000196ca0, 0x0, 0x0, 0x0)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/yaml.go:161 +0x270
github.com/canonical/go-dqlite/vendor/gopkg.in/yaml%2ev3.Unmarshal(...)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/vendor/gopkg.in/yaml.v3/yaml.go:89
github.com/canonical/go-dqlite/app.fileUnmarshal(0xc00046aa80, 0x1e, 0x15cf6b8, 0x9, 0x1512620, 0xc000196ca0, 0x0, 0x127ed30)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/app/files.go:71 +0x174
github.com/canonical/go-dqlite/app.New(0xc00046aa80, 0x1e, 0xc000196c80, 0x3, 0x4, 0xc000196c80, 0x0, 0x0)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/app/app.go:113 +0x2218
github.com/canonical/go-dqlite/app_test.newAppWithDir(0xc0003fc000, 0xc00046aa80, 0x1e, 0xc00011df58, 0x1, 0x1, 0x0, 0xc000481d00)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/app/app_test.go:1079 +0x190
github.com/canonical/go-dqlite/app_test.TestNew_JoinerRestart(0xc0003fc000)
	/<<PKGBUILDDIR>>/obj-s390x-linux-gnu/src/github.com/canonical/go-dqlite/app/app_test.go:79 +0x2da
testing.tRunner(0xc0003fc000, 0x15edc28)
	/usr/lib/go-1.13/src/testing/testing.go:909 +0xd6
created by testing.(*T).Run
	/usr/lib/go-1.13/src/testing/testing.go:960 +0x36c
FAIL	github.com/canonical/go-dqlite/app	0.698s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants