Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add convenient packaging infrastructure #4

Merged
merged 3 commits into from
Jan 10, 2020

Conversation

amotl
Copy link
Contributor

@amotl amotl commented Jan 9, 2020

Introduction

By making a start to #3 in order to bring this program to a wider audience, this adds a lightweight but convenient packaging infrastructure.

Goals

  • We should provide ready-to-go packages for popular *nix operating systems on the GitHub release page. However, this should be as effortless as possible. Nobody has time to write and maintain individual packagers supporting the plethora of Linux/Unix variants out there.
  • Providing armhf packages alongside amd64 will unlock support for all kinds of Single-board computers like BeagleBone, RaspberryPi and friends.

History

The package building subsystem was originally conceived for Kotori the other day. ocli now has the pleasure of profiting from a trimmed and generalized version of it.

Outlook

During cherry-picking the gems over from the Kotori repository, I tried to make most spots as generic as possible, so that it can be extended to other operating systems and architectures. Also, @jpmens and @ckrey might want to reuse it 1:1 for qtripp and friends as it works almost plug & play now.

@amotl amotl force-pushed the packaging branch 3 times, most recently from 4ec3f8b to dc98ebd Compare January 9, 2020 02:43
@amotl amotl mentioned this pull request Jan 9, 2020
@amotl amotl changed the title Add ready-to-go infrastructure for Docker-based packaging Add convenient packaging infrastructure Jan 9, 2020
@amotl amotl force-pushed the packaging branch 2 times, most recently from d1dba9e to aa9558f Compare January 9, 2020 03:13
@amotl
Copy link
Contributor Author

amotl commented Jan 9, 2020

About

The packaging process is completely automated based on Docker and fpm, which can build packages for the following targets:

  • deb
  • rpm
  • solaris
  • freebsd
  • tar
  • directories
  • Mac OS X .pkg files (osxpkg)
  • pacman (ArchLinux) packages

Details

  • Currently, the infrastructure has been prepared to build Debian packages for both amd64 and armhf architectures through corresponding make targets.
  • Whether you would like to build against Debian 9.x stretch or Debian 10.x buster is completely up to you. This can be controlled by providing respective arguments on the command line, see "Usage" section.

Usage

Read more about its usage within Packaging README.

Demo

https://github.com/daq-tools/owntracks-publisher/releases

Backlog

There are some more things to polish. We are tracking them at #5.

@jpmens
Copy link
Member

jpmens commented Jan 10, 2020

@amotl thank you for this contribution.

The changes to the Makefile mean that it no longer builds on non-gmake systems, so I won't take it as it stands; the packaging bits will have to be detached from the Makefile, probably best into their own independent packaging/Makefile?

In order to bring this program to a wider audience,
this adds a lightweight but convenient packaging
infrastructure.

Read more about its usage within packaging/README.rst.
@amotl
Copy link
Contributor Author

amotl commented Jan 10, 2020

Dear @jpmens,

I changed the packaging to work from within the packages directory as you suggested. There, it has a separate Makefile for doing its work.

With kind regards,
Andreas.

@jpmens jpmens merged commit c15d590 into owntracks:master Jan 10, 2020
@jpmens
Copy link
Member

jpmens commented Jan 10, 2020

Merged, thank you

@amotl
Copy link
Contributor Author

amotl commented Jan 14, 2020

@jpmens

The packaging process is completely automated based on Docker and fpm.

I see that you have been using fpm already.

So, I recognize this is actually nothing new to your toolbox. However, the Docker integration now makes it possible to build that jazz without even having the respective Linux distribution installed on a real machine.

@jpmens
Copy link
Member

jpmens commented Jan 15, 2020

@amotl I didn't know that, thanks.

@amotl
Copy link
Contributor Author

amotl commented Jan 15, 2020

The Docker integration with fpm now makes it possible to build that jazz without even having the respective Linux distribution installed on a real machine.

I didn't know that, thanks.

Just try to invoke these commands [1] on your workstation. They should work out of the box. If not, please let me know.

[1] https://github.com/owntracks/ocli/blob/master/packaging/README.rst

@jpmens
Copy link
Member

jpmens commented Jan 15, 2020

Be warned: I have no idea what's going on:

$ docker login
Username: jpmens
Password: xx
Login Succeeded

$ make  debian-package arch=amd64 dist=buster version=0.7.0
/Applications/Xcode.app/Contents/Developer/usr/bin/make deb-build name=owntracks-publisher
Building package owntracks-publisher version 0.7.0 for architecture amd64 and distribution buster.
docker build --tag owntracks/owntracks-publisher-build-buster-amd64:0.7.0 --build-arg BASE_IMAGE=owntracks/baseline-buster-amd64:latest --build-arg DISTRIBUTION=buster --build-arg VERSION=0.7.0 --build-arg NAME=owntracks-publisher --file dockerfiles/Dockerfile.builder ..
Sending build context to Docker daemon  5.353MB
Step 1/20 : ARG BASE_IMAGE
Step 2/20 : FROM ${BASE_IMAGE} AS build-environment
pull access denied for owntracks/baseline-buster-amd64, repository does not exist or may require 'docker login'
make[1]: *** [deb-build] Error 1
make: *** [debian-package] Error 2

@amotl
Copy link
Contributor Author

amotl commented Jan 15, 2020

@jpmens Please run the step for preparing the baseline images [1] on your local workstation. docker login is not required at all.

[1] https://github.com/owntracks/ocli/blob/master/packaging/README.rst#setup

@jpmens
Copy link
Member

jpmens commented Jan 15, 2020

Wer lesen kann ist klar im Vorteil ...

@jpmens
Copy link
Member

jpmens commented Jan 15, 2020

Who's GITHUB_TOKEN ist that? (642ff7c47b1697a79ab7f105e1d79f054d0bbeef)

@jpmens
Copy link
Member

jpmens commented Jan 15, 2020

I assume I export mine.

@amotl
Copy link
Contributor Author

amotl commented Jan 15, 2020

Who's GITHUB_TOKEN ist that? (642ff7c47b1697a79ab7f105e1d79f054d0bbeef)

It's a completely random one. You might want to use your own token which has access to owntracks/ocli. For testing this task, my personal GITHUB_TOKEN worked on daq-tools/owntracks-publisher well, see [1].

[1] https://github.com/daq-tools/owntracks-publisher/releases/tag/0.7.0

@amotl
Copy link
Contributor Author

amotl commented Jan 15, 2020

make debian-package arch=amd64 dist=buster version=0.7.0

You might want to bump that to version=0.8.0 already.

@jpmens
Copy link
Member

jpmens commented Jan 15, 2020

I'm not that thick ;-)
Yes, done that.

@jpmens
Copy link
Member

jpmens commented Jan 15, 2020

jmbp-5947

OMG.

@jpmens
Copy link
Member

jpmens commented Jan 15, 2020

@amotl you realize what you've caused, don't you? I've smelled blood, and I now want that magical Makefile all over the show, and can we add CentOS? :-)

@amotl
Copy link
Contributor Author

amotl commented Jan 15, 2020

Building for CentOS

Will that work in general?

CentOS, difficult as dependencies (gpsd, mosquitto) currently not in Epel8. See #5.

I actually have been able to satisfy these dependencies and make it build already.

# From: https://centos.pkgs.org/7/epel-x86_64/gpsd-3.10-5.20140524gitd6b65b.el7.x86_64.rpm.html
# TODO: Adjust for CentOS 8.
RUN wget https://download-ib01.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-12.noarch.rpm
RUN rpm -Uvh epel-release*rpm

RUN yum install -y gpsd-devel mosquitto-devel

Outcome

While building for CentOS 6 didn't invoke at all (can't remember the reason), It croaked for me later in the packaging process for both the amd64 and arm64v8 architecture targets on CentOS 7. I believe it was on the invocation of fpm. Haven't tried CentOS 8 though, but you will be able to adjust easily.

Preview

Build baseline image

# CentOS 7
make build-centos-baseline arch=amd64 dist=7 version=0.1.0
# defunct; make build-centos-baseline arch=arm32v7 dist=7 version=0.1.0
make build-centos-baseline arch=arm64v8 dist=7 version=0.1.0

Invoke packaging

# CentOS 7 amd64
make centos-package arch=amd64 dist=centos7 pkgtype=rpm version=0.7.0

I will add general packaging support for CentOS right away. However, it still has the problems outlined within this post.

@amotl amotl mentioned this pull request Jan 15, 2020
@jpmens
Copy link
Member

jpmens commented Jan 15, 2020

No hurry, no rush. Thank you. :)

@jpmens
Copy link
Member

jpmens commented Jan 15, 2020

When you're done I will rename the package to

DIST_PACKAGE := owntracks-cli-publisher

to make it homogenous with the repo name.

@amotl
Copy link
Contributor Author

amotl commented Jan 15, 2020

It croaked for me later in the packaging process for both the amd64 and arm64v8 architecture targets on CentOS 7. I believe it was on the invocation of fpm.

Maybe it was because the parameters of the fpm-package helper [1] have not been adjusted appropriately, like seen within [2].

[1] https://github.com/owntracks/ocli/blob/0.8.0/packaging/builder/fpm-package
[2] https://github.com/owntracks/recorder/blob/0.8.4/etc/centos/fpm-make.sh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants