ci: nDPI build issue inside drone-run fedora:34 #1

New Issue

wanderer · 2021-04-14T00:18:56+02:00

wanderer commented

2021-04-14 00:18:56 +02:00

nDPI builds fail on the ./configure step with

...
58	checking how to print strings... printf 29s
59	checking whether make supports the include directive... yes (GNU style) 29s
60	checking for gcc... gcc 29s
61	checking whether the C compiler works... no 29s
62	configure: error: in `/drone/nDPI': 29s
63	configure: error: C compiler cannot create executables 29s
64	See `config.log' for more details 29s

⇒ https://drone.dotya.ml/wanderer/fastnetmon-ng/51/3/4

what's curious is that:

I haven't been able to reproduce this in a standalone fedora:34 docker container under:
- fedora 34
- archlinux
this doesn't happen on archlinux at all (see archlinux build log)
⇒ https://drone.dotya.ml/wanderer/fastnetmon-ng/51/2/4

nDPI builds fail on the `./configure` step with ```log ... 58 checking how to print strings... printf 29s 59 checking whether make supports the include directive... yes (GNU style) 29s 60 checking for gcc... gcc 29s 61 checking whether the C compiler works... no 29s 62 configure: error: in `/drone/nDPI': 29s 63 configure: error: C compiler cannot create executables 29s 64 See `config.log' for more details 29s ``` ⇒ https://drone.dotya.ml/wanderer/fastnetmon-ng/51/3/4 what's curious is that: * I haven't been able to reproduce this in a standalone `fedora:34` docker container under: * fedora 34 * archlinux * this doesn't happen on archlinux *at all* (see archlinux build log) ⇒ https://drone.dotya.ml/wanderer/fastnetmon-ng/51/2/4

wanderer added the

bug

label 2021-04-14 00:18:56 +02:00

wanderer self-assigned this 2021-04-14 00:18:56 +02:00

wanderer commented

2021-04-20 11:25:59 +02:00

This has been at first misdiagnosed as an nDPI issue.
What has really been going on was that due to the nature of drone's atomic step execution, there was no knowledge preserved of the step before inside of the container running the step after (that's kind of the feature of the docker ephemeral pipelines with each step being run in a fresh container, could have thought of that...).

That is, at first deps were installed into the image (fedora:34) required for building both nDPI and fastnetmon, then in the following steps they were to be utilised, naturally, but since they were installed in /bin, some configs in /etc, these were not preserved and a fresh image was used in the next step.

A working solution appeared to be a little ugly - telling drone to preserve the whole /usr directory (of which /bin ---> /usr/bin is a part of) as a temporary "cache" volume and mounting that into every pipeline step that required it.

This still wasn't enough since we installed both lld and binutils packages, both coming with a linked. Programs, however, expect to find just /bin/ld and in case of multiple linkers installed, on fedora this is transparently managed by dnf (the package manager) that symlinks one of the linkers to its expected place (/bin/ld)...from /etc/alternatives...whatever.

To come around this issue, /etc has also had to be mounted as a "cache" volume in pipeline steps that required a linker.
⇒ https://git.dotya.ml/wanderer/fastnetmon-ng/src/branch/cmakelists-pimp-up/.drone.yml#L178-L182

First successful build since introducing the above mentioned approach:
⇒ https://drone.dotya.ml/wanderer/fastnetmon-ng/89/3/3

Commit fixing this:
⇒ 8dd5fdabe4

This has been at first misdiagnosed as an `nDPI` issue. What has really been going on was that due to the nature of drone's atomic step execution, there was no knowledge preserved of the step before inside of the container running the step after (that's kind of the feature of the docker ephemeral pipelines with each step being run in a fresh container, could have thought of that...). That is, at first deps were installed into the image (`fedora:34`) required for building both nDPI and fastnetmon, then in the following steps they were to be utilised, naturally, but since they were installed in `/bin`, some configs in `/etc`, these were not preserved and a fresh image was used in the next step. A working solution appeared to be a little ugly - telling drone to preserve the whole `/usr` directory (of which `/bin` ---> `/usr/bin` is a part of) as a temporary "cache" volume and mounting that into every pipeline step that required it. This still wasn't enough since we installed both `lld` and `binutils` packages, both coming with a linked. Programs, however, expect to find just `/bin/ld` and in case of multiple linkers installed, on fedora this is transparently managed by `dnf` (the package manager) that symlinks one of the linkers to its expected place (`/bin/ld`)...from `/etc/alternatives...whatever`. To come around this issue, `/etc` has also had to be mounted as a "cache" volume in pipeline steps that required a linker. ⇒ https://git.dotya.ml/wanderer/fastnetmon-ng/src/branch/cmakelists-pimp-up/.drone.yml#L178-L182 First successful build since introducing the above mentioned approach: ⇒ https://drone.dotya.ml/wanderer/fastnetmon-ng/89/3/3 Commit fixing this: ⇒ https://git.dotya.ml/wanderer/fastnetmon-ng/commit/8dd5fdabe4e94d6d036b7eb944f23301924793bd

wanderer closed this issue

2021-04-20 11:25:59 +02:00

wanderer added this to the General project 2021-05-04 23:28:27 +02:00

Sign in to join this conversation.