Fixup date
[blog.git] / _posts / 2020-11-01-debian-new-queue,-rust-packaging.md
1 ---
2 layout: post
3 title: Debian NEW Queue, Rust packaging
4 categories:
5 - debian
6 - ftpmaster
7 tags:
8 - NEW
9 - rust
10 date: '2020-11-03 00:38:00 +0100'
11 ---
12 # Debian NEW Queue
13
14 So for some reason I got myself motivated again to deal with some
15 packages in Debians [NEW Queue](https://ftp-master.debian.org/new.html).
16 We had 420 source packages waiting for some kind of processing when I
17 started, now we are down to something [around
18 10](https://ftp-master.debian.org/stat.html). (Silly, people keep
19 uploading stuff...)
20
21 That's not entirely my own work, others from the team have been active
22 too, but for those few days I went through a lot of stuff waiting. And
23 must say it still feels mostly like it did when I somehow stopped
24 doing much in NEW.
25
26 Except - well, I **feel** that maintainers are much better in preparing
27 their packages, especially that dreaded task of getting the copyright
28 file written seems to be one that is handled much better. Now, thats
29 not supported by any real numbers, just a feeling, but a good one, I
30 think.
31
32
33 # Rust
34
35 Dealing with NEW meant I got in contact with one part that currently
36 generates some friction between the FTP Team and one group of package
37 maintainers - the Rust team.
38
39 Note: this is, of course, entirely written from my point of view.
40 Though with the intention of presenting it as objective as possible.
41 Also, I know what rust is, and have tried a "Hello world" in it, but
42 that's about my *deep* knowledge of it...
43
44
45 ## The problem
46
47 Libraries in rust are bundled/shipped/whatever in something called
48 crates, and you manage what your stuff needs and provides with a tool
49 called *cargo*.
50
51 A library (one per crate) can provide multiple features, say a TLS lib
52 can link against gnutls or openssl or some other random
53 implementation. Such features may even be combinable in various
54 different ways, so one can have a high number of possible feature
55 combinations for one crate.
56
57 There is a tool called {% deb_pkg debcargo %} which helps creating a
58 Debian package out of a crate. And that tool generates so-called
59 ***feature-packages***, one per feature / combination thereof.
60
61 Those feature packages are empty packages, only containing a symlink
62 for their */usr/share/doc/...* directory, so their size is smaller
63 than the metadata they will produce. Inside the archive and the files
64 generated by it, stuff that every user everywhere has to download
65 and their apt has to process. Additionally, any change of those feature
66 sets means one round through NEW, which is also not ideal.
67
68
69 So, naturally, the FTP Team dislikes those empty feature packages.
70 Really, a lot.
71
72 There appears to be a different way. Not having the feature packages,
73 but putting all the combinations into a *Provides* header. That
74 sometimes works, but has two problems:
75
76 - It can generate really long Provides: lines. I mean, REALLY
77 **REALLY** ***REALLY*** long. Somewhat around 250kb is the current
78 record. Thats long enough that a tool (not dak itself) broke on it.
79 Sure, that tool needs to be fixed, but still, that's not nice.
80 Currently preferred from us, though.
81 - Some of the features may need different dependencies (say, gnutls
82 vs openssl), should those conflict with each other, you can not
83 combine them into one package.
84
85 ## Solutions
86
87 Currently we do not have a good one. The rust maintainers and the ftp
88 team are talking, exploring various ideas, we will see what will come
89 out.
90
91
92 # Devel archive / Component
93
94 One of the possible solutions for the feature package problem would be
95 something that another set of packages could also make good use of, I
96 think. The introduction of a new archive or component, meant only for
97 packages that are needed to build something, but where users are
98 discouraged from ever using them.
99
100 _What_?
101
102 Well, take golang as an example. While we have a load of
103 golang-something packages in Debian, and they are used for building
104 applications written in go - none of those golang-something are meant
105 to be installed by users. If you use the language and develop in it,
106 the **go get** way is the one you are expected to use.
107
108 So having an archive (or maybe component like main or contrib) that,
109 by default, won't be activated for users, but only for things like
110 buildds or archive rebuilds, will make one problem (hated metadata
111 bloat) be evaluated wildly different.
112
113 It may also allow a more relaxed processing of binary-NEW (easier
114 additions of new feature packages).
115
116 ## But but but
117
118 Yes, it is not the most perfect solution. Without taking much energy
119 to think about, it requires
120 - an adjustment in how main is handled. Right now we have the golden
121 rule that *main is self contained*, that is, things in it may not
122 need anything outside it for building or running. That would need
123 to be adjusted for building. (Go as well as *currently* rust are
124 always building static binaries, so no library dependencies there).
125 - It would need handling for the release, that is, the release team
126 would need to deal with that archive/component too. We haven't,
127 yet, talked to them (still, slowly, discussing inside FTP Team).
128 So, no idea how many rusty knives they want to sink into our nice
129 bodies for that idea...
130
131 # Final
132
133 Well, it is still very much open. Had an IRC meeting with the rust
134 people, will have another end of November, it will slowly go forward.
135 And maybe someone comes up with an entire new idea that we all love.
136 Don't know, time will tell.