Add frontend/backend make targets, fix source release

[silverwind]

• Fix missing 'dist' folders inside 'node_modules' being excluded because of tar excluding them recursively.

• Add 'make build-offline' which skips on verifying node_modules and stops npx from downloading missing dependencies on-demand.

• Store VERSION in file VERSION for the release tarballs and prefer that file over git-derived version.

[codecov-io]

Codecov Report

Merging #10325 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #10325      +/-   ##
==========================================
- Coverage   43.75%   43.74%   -0.02%     
==========================================
  Files         586      586              
  Lines       81265    81265              
==========================================
- Hits        35560    35549      -11     
- Misses      41309    41317       +8     
- Partials     4396     4399       +3

Impacted Files	Coverage Δ
services/pull/check.go	50% <0%> (-3.05%)	⬇️
modules/git/command.go	86.95% <0%> (-2.61%)	⬇️
models/notification.go	63.29% <0%> (-1.9%)	⬇️
modules/queue/workerpool.go	58% <0%> (+1.06%)	⬆️
modules/indexer/stats/db.go	59.37% <0%> (+9.37%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c8d1c38...27039e9. Read the comment docs.

[silverwind]

Maybe we need a quick vote:

👍 ship built webpack assets in the source tarball
👎 don't ship built webpack assets in the source tarball

Decision made, see #10325 (comment)

[guillep2k]

Maybe we need a quick vote:

👍 ship built webpack assets in the source tarball
👎 don't ship built webpack assets in the source tarball

I guess it depends. As long as it's still useful for the package distributions, I figure the smaller the better. But I can't really say what "strict rules" they need to abide. 😓

Please take this with a grain of salt since packages and distributions are not my area of expertise: I think it would be cool to leave only go code left for compiling, with all assets ready to pack (css, js, etc). That way anyone's offline-built Gitea would be as close to our own packages as it gets. If any intermediate node stages could give us a surprise in one platform or another, that's avoided (*). We should probably not include anything node related (save of course source code directly from our repo) since anybody else that just wants the tarball for building from source can actually go and download those.

(*) We've recently turned down a js library because it referenced pre-compiled binaries that were not available in all platforms; I can't remember what it was ATM.

[silverwind]

Yeah I guess it's a matter of "we compile" vs. "user compiles". But given that the target is the browser, it really does not matter who does it and I think we're in a better position to produce proper files then relying on a inconsistent Node.js version installed by the distributor on their build system. Still Node.js should stay as a build dependency of course, even if not used in this special case.

Native Node.js dependencies are a absolute no-go and I think we should actually enforce a check for it on CI, maybe something I will check out later.

I will update the PR to include the assets again later.

[silverwind]

This will need a 1.11 backport to fully resolve #10253.

[silverwind]

So the only remaining topic here is whether we should attempt to skip the fomantic/webpack build because depending on tar extraction order, the webpack and fomantic steps will run on a extracted source tarball that includes the assets. We could potentially add a hackish @touch $(WEBPACK_DEST) $(FOMANTIC_EVIDENCE) prereq step to the build-offline but I'd prefer to not add tarball-specific hacks to that target.

[guillep2k]

So the only remaining topic here is whether we should attempt to skip the fomantic/webpack build

👍

I vote for skipping them. If we reorder the dependency tree (maybe split a couple of them in two) we should be able to remove those from the list. i.e. make build-offline should only require go.

[silverwind]

If I remove them from the prereq chain, make build-offline does not seem like a appropriate name any more, as it's now make build-offline-without-frontend-assets. Any idea on a better name? Maybe make build-dist?

[guillep2k]

Maybe make build-dist?

Sounds good to me!

[sapk]

build-from-archive ?

[silverwind]

PR reworked, here is a updated summary:

- Add 'make backend' and 'make frontend' make targets which are used to
  build go and js/css/svg files respectively.

- The 'backend' target can be invoked without requiring Node.js to be
  present on the system if pre-built frontend assets are present like
  in the release source tarballs.

- Fix source releases missing 'dist' folders inside 'node_modules' which
  were erronously excluded from tar.

- Store VERSION in file VERSION for the release tarballs and prefer that
  file over git-derived version.

[guillep2k]

Wow, this looks much nicer! 🚀

[lunny]

Please send backport to v1.11

[silverwind]

Backport: #10414

[strk]

Sorry, I'm confused: in current master branch (6b01972) I find no build-offline nor backend Makefile target. What's the deal ?

[jolheiser]

6b01972 is from 6 days ago, this was merged after that

[strk]

On Thu, Feb 27, 2020 at 08:26:40AM -0800, John Olheiser wrote: <6b01972> is from 6 days ago

I'm running commit a924a90 now, and `backend` target there, thanks (build-offline isnt' but it's ok)

[silverwind]

build-offline never made it to any official branch, fyi.