Arch Linux

> Can anobody tell me, how we apply patches to downloaded artifacts while building?!

I only have a racy solutions for that. Put the following at the start of build():

while true; do
find $rust-build-dir-path -type f -name $name-to-patch -execdir patch -p1 -i $srcdir/patch \; && break
done &

trying to compile rust 1.34 with llvm-libs 7:

error[E0428]: the name `U1024` is defined multiple times
    --> /build/rust/src/rustc-1.34.0-src/build/i686-unknown-linux-gnu/stage2-tools/i686-unknown-linux-gnu/release/build/typenum-101079a7a9ba8c17/out/consts.rs:2112:5
     |
2110 |     pub type U1024 = UInt, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>;
     |     ------------------------------------------------------------------------------------------------------------------------------------- previous definition of the type `U1024` here
2111 |     pub type P1024 = PInt; pub type N1024 = NInt;
2112 |     pub type U1024 = UInt, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>;
     |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `U1024` redefined here
     |
     = note: `U1024` must be defined only once in the type namespace of this module

error[E0428]: the name `P1024` is defined multiple times
    --> /build/rust/src/rustc-1.34.0-src/build/i686-unknown-linux-gnu/stage2-tools/i686-unknown-linux-gnu/release/build/typenum-101079a7a9ba8c17/out/consts.rs:2113:5
     |
2111 |     pub type P1024 = PInt; pub type N1024 = NInt;
     |     ----------------------------- previous definition of the type `P1024` here
2112 |     pub type U1024 = UInt, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>;
2113 |     pub type P1024 = PInt; pub type N1024 = NInt;
     |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `P1024` redefined here
     |
     = note: `P1024` must be defined only once in the type namespace of this module

error[E0428]: the name `N1024` is defined multiple times
    --> /build/rust/src/rustc-1.34.0-src/build/i686-unknown-linux-gnu/stage2-tools/i686-unknown-linux-gnu/release/build/typenum-101079a7a9ba8c17/out/consts.rs:2113:35
     |
2111 |     pub type P1024 = PInt; pub type N1024 = NInt;
     |                                   ----------------------------- previous definition of the type `N1024` here
2112 |     pub type U1024 = UInt, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>;
2113 |     pub type P1024 = PInt; pub type N1024 = NInt;
     |                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `N1024` redefined here
     |
     = note: `N1024` must be defined only once in the type namespace of this module

error: aborting due to 3 previous errors

let's try again with:

  while true; do
    sed -i '
      2112 { /pub type U1024/ d }
      2113 { /pub type P1024/ d }
    ' build/i686-unknown-linux-gnu/stage2-tools/i686-unknown-linux-gnu/release/build/typenum-101079a7a9ba8c17/out/consts.rs 2>/dev/null || true
  done &

injected in build() ... which is the mother of all race conditions

current error:

error: this file contains an un-closed delimiter
   --> /build/rust/src/rustc-1.34.0-src/build/i686-unknown-linux-gnu/stage2-tools/i686-unknown-linux-gnu/release/build/typenum-101079a7a9ba8c17/out/consts.rs:482:56
    |
54  | pub mod consts {
    |                - un-closed delimiter
...
482 |     pub type U210 = UInt`, const, identifier, lifetime, or type, found `}`
   --> /build/rust/src/rustc-1.34.0-src/build/i686-unknown-linux-gnu/stage2-tools/i686-unknown-linux-gnu/release/build/typenum-101079a7a9ba8c17/out/consts.rs:482:55
    |
482 |     pub type U210 = UInt`, const, identifier, lifetime, or type here

error: aborting due to 2 previous errors

error: Could not compile `typenum`.
warning: build failed, waiting for other jobs to finish...
error: build failed

trying again with some even-more-racing build()

build() {
  cd "rustc-$pkgver-src"

  while true; do
    find build/i686-unknown-linux-gnu/stage2-tools/i686-unknown-linux-gnu/release/build \
      -type f \
      -name consts.rs \
      -exec grep -qF 'pub type P1024' {} \; \
      -exec sleep 1 \; \
      -exec sed -i '
        2112 { /pub type U1024/ d }
        2113 { /pub type P1024/ d }
      ' {} \; 2>/dev/null || true
  done &
  _kill_pid=$!

  python2 ./x.py build -j"$(nproc)"
  kill $_kill_pid
}

this is super-annoying: now the old "duplicate definition" error is back ... but in package_rust()
Let's build again (for several hours O.o) with this hack also applied in package_rust() ...

whoah, even more fun:

warning: the feature `try_from` has been stable since 1.34.0 and no longer requires an attribute to enable
  --> src/tools/clippy/clippy_lints/src/lib.rs:12:12
   |
12 | #![feature(try_from)]
   |            ^^^^^^^^
   |
   = note: #[warn(stable_features)] on by default

warning: the feature `try_from` has been stable since 1.34.0 and no longer requires an attribute to enable
  --> src/tools/clippy/clippy_lints/src/lib.rs:12:12
   |
12 | #![feature(try_from)]
   |            ^^^^^^^^
   |
   = note: #[warn(stable_features)] on by default

warning: the feature `try_from` has been stable since 1.34.0 and no longer requires an attribute to enable
 --> src/tools/clippy/src/driver.rs:4:12
  |
4 | #![feature(try_from)]
  |            ^^^^^^^^
  |
  = note: #[warn(stable_features)] on by default

    Finished release [optimized] target(s) in 9m 17s
Building stage2 tool rls (i686-unknown-linux-gnu)
error: the listed checksum of `/build/rust/src/rustc-1.34.0-src/vendor/rustc-ap-rustc_target/spec/i686_unknown_linux_gnu.rs` has changed:
expected: a75a6025d7e3424edf9baf3039056c0f8eea157631a175d00ac5a218aa54b510
actual:   484bf8be15015b330fa9a97b6dabb8c7627e59d5cddb2dd0e83478749f8aabad

directory sources are not intended to be edited, if modifications are required then it is recommended that [replace] is used with a forked copy of the source
command did not execute successfully: "/usr/bin/cargo" "build" "--target" "i686-unknown-linux-gnu" "-j" "4" "--release" "--frozen" "--manifest-path" "/build/rust/src/rustc-1.34.0-src/src/tools/rls/Cargo.toml" "--features" "clippy" "--message-format" "json"
expected success, got: exit code: 101
thread 'main' panicked at 'Unable to build RLS', src/bootstrap/dist.rs:65:9
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
failed to run: /build/rust/src/rustc-1.34.0-src/build/bootstrap/debug/bootstrap install
Build completed unsuccessfully in 2:02:38

current approach: using rust-1.36 from manjaro32 (thanks, jonathon!) ... this fails in stage 1, because it does not find some library, which actually _is_ there (but apparently not in the search path).

symlinking the library to the correct one hidden in some "stage1" folder solved it. Now we have rust 1.36 for i686 in [build-support] - let's hope some build slave can build a regular rust (e.g. one where I do not extract additional files with bsdtar into the build chroot) from that and we can close this bug report.

Rebuilds of rust on i686 and pentium4 show:

--- stderr
error: couldn't load codegen backend "/usr/lib/rustlib/i686-unknown-linux-gnu/codegen-backends/librustc_codegen_llvm-llvm.so": "libLLVM-7.so: cannot open shared object file: No such file or directory"

Tried with precompiled binaries (AUR package 'rust-bin'):

https://aur.archlinux.org/packages/rust-bin/

Compiling 1.37.0 from source now results in LLVM: out of memory,
see also:

https://github.com/rust-lang/rust/issues/60294

Fedora 31 will also complain, so there is a chance Rust people are actually doing something
and not just ignoring us:

https://bugzilla.redhat.com/show_bug.cgi?id=1723064

Asked upstream about how rust is supposed to be built on a machine with 32-bit
address space:

https://github.com/rust-lang/rust/issues/60294

–debuginfo-level-std=1

in config.toml

works over the first out of memory situations.

Now I get:

error: failed to run custom build command for `backtrace-sys v0.1.27`

Caused by:

process didn't exit successfully: `/build/rust/src/rustc-1.37.0-src/build/i686-unknown-linux-gnu/stage2-std/release/build/backtrace-sys-e11a26709e2be131/build-script-build` (exit code: 101)

— stdout

command did not execute successfully: "/usr/bin/cargo" "build" "–target" "x86_64-unknown-linux-gnu" "-j" "1" "–release" "–frozen" "–features" "panic-unwind backtrace profiler" "–manifest-path" "/build/rust/src/rustc-1.37.0-src/src/libstd/Cargo.toml" "–message-format" "json"
expected success, got: exit code: 101

', /build/rust/src/rustc-1.37.0-src/vendor/cc/src/lib.rs:2398:5
stack backtrace:

 0: std::panicking::default_hook::{{closure}}
 1: std::panicking::default_hook
 2: std::panicking::rust_panic_with_hook
 3: std::panicking::continue_panic_fmt
 4: std::panicking::begin_panic_fmt
 5: cc::fail
           at /build/rust/src/rustc-1.37.0-src/vendor/cc/src/lib.rs:2398
 6: cc::Build::compile
           at /build/rust/src/rustc-1.37.0-src/vendor/cc/src/lib.rs:951
 7: build_script_build::main
           at ./build.rs:107
 8: std::rt::lang_start::{{closure}}
 8: std::rt::lang_start::{{closure}}
           at /rustc/eae3437dfe991621e8afdc82734f4a172d7ddf9b/src/libstd/rt.rs:64

note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

I'm personally quite fed up with the quality and documentation of this compiler..

cargo:warning=cc1: sorry, unimplemented: 64-bit mode not compiled in

Does this mean that the architecture triplet is wrong?

Indeed. Fixed config.toml, which was completely wrong ([build] had a amd64 target,
building for two architectures instead one (i686)..

pentium4 went through, trying now the sed trick above on the i686 version..

error[E0428]: the name `U1024` is defined multiple times
    --> /build/rust/src/rustc-1.37.0-src/build/i686-unknown-linux-gnu/stage2-tools/i686-unknown-linux-gnu/release/build/typenum-895469ec3d111e10/out/consts.rs:2112:5
     |
2110 |     pub type U1024 = UInt, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>;
     |     ------------------------------------------------------------------------------------------------------------------------------------- previous definition of the type `U1024` here
2111 |     pub type P1024 = PInt; pub type N1024 = NInt;
2112 |     pub type U1024 = UInt, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>;
     |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `U1024` redefined here
     |
     = note: `U1024` must be defined only once in the type namespace of this module

error[E0428]: the name `P1024` is defined multiple times
    --> /build/rust/src/rustc-1.37.0-src/build/i686-unknown-linux-gnu/stage2-tools/i686-unknown-linux-gnu/release/build/typenum-895469ec3d111e10/out/consts.rs:2113:5
     |
2111 |     pub type P1024 = PInt; pub type N1024 = NInt;
     |     ----------------------------- previous definition of the type `P1024` here
2112 |     pub type U1024 = UInt, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>;
2113 |     pub type P1024 = PInt; pub type N1024 = NInt;
     |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `P1024` redefined here
     |
     = note: `P1024` must be defined only once in the type namespace of this module

error[E0428]: the name `N1024` is defined multiple times
    --> /build/rust/src/rustc-1.37.0-src/build/i686-unknown-linux-gnu/stage2-tools/i686-unknown-linux-gnu/release/build/typenum-895469ec3d111e10/out/consts.rs:2113:35
     |
2111 |     pub type P1024 = PInt; pub type N1024 = NInt;
     |                                   ----------------------------- previous definition of the type `N1024` here
2112 |     pub type U1024 = UInt, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>, B0>;
2113 |     pub type P1024 = PInt; pub type N1024 = NInt;
     |                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `N1024` redefined here
     |
     = note: `N1024` must be defined only once in the type namespace of this module

error: aborting due to 3 previous errors

For more information about this error, try `rustc --explain E0428`.
   Compiling strip-ansi-escapes v0.1.0
error: Could not compile `typenum`.

There is no documentation how to use x.py properly, what the build does, what
the bootstrapping does. And life-patching of typenum is just the really last
straw (currently failing)..

Currently blocking:
- librsvg2 on i686 (rust-bin generates SSE2-code)
- firefox, thunderbird and friends on i686
- some random software on i686 like newsboat

We should try to understand x.py

https://github.com/rust-lang/rust/blob/master/src/bootstrap/README.md

To me it looks like we can have a PKGBUILD calling each stage individually and do
the necessary patching in-between.

Ok, I see several issues here:

rustc –target i686-unknown-linux-gnu –print cfg

says:

target_feature="sse"
target_feature="sse2"

rustc –target i586-unknown-linux-gnu –print cfg

has no SSE/SSE2, so we pick:
- i686-unknown-linux-gnu for our pentium4 and
- i586-unknown-linux-gnu for our i486 and i686

see also some interesting notes in:
- https://github.com/rust-lang/rust/issues/54740 - https://internals.rust-lang.org/t/is-pentium4-still-an-appropriate-base-cpu-for-i686/7353/6

The current config.toml.patch doesn't apply correctly and silently fails:

patching file config.toml
Hunk #1 FAILED at 2.
Hunk #2 succeeded at 21 with fuzz 2 (offset 2 lines).
1 out of 2 hunks FAILED -- saving rejects to file config.toml.rej

This needs proper treatment in the build scripts, as we cannot continue building
if patches apply partially!

Now I get the same typenum error also in stage2 of cargo on pentium4.

Trying the stage 2 –keep-stage 1 trick..

Some internal documentation on x.py:

https://rust-lang.github.io/rustc-guide/how-to-build-and-run.html

Cool, so the conflicting consts.rs of typenum appears out of the blue in the middle of a stage.
So this multi-layered building will also not work.

Trying an inotifywait approach watching for consts.rs to appear and calling a sed on it then:

inotifywait -mrq -e create --format %w%f $srcdir | while read -r FILE; do
  case "$$FILE" in
    *consts.rs)
      echo "--> patching $$FILE"
      sed -i '/pub type U1024/d;/pub type P1024/d' $FILE
  esac
done

The inotify-trick helped to rebuild 1.38 for pentium4.

So, rust 1.38 can only be built with rust 1.37, not with 1.38 (rebuilding itself),
see:

https://github.com/rust-lang/rust/issues/63911

This means, having a rust package building rust works, if you do it once,
otherwise I would suggest to upstream to have a rust137 and build rust138
which provides rust, so the build can be done more than once (especially
automatically).

When I use rust-bin for 1.37 I cannot build 1.38 again for i686 (out of
memory). This one I'm trying to tackle with debug_level=0 (which seems
to help). Didn't have this problem with 'rust' 1.37.0 on pentium4 for
rebuilding 1.38.0, so maybe rust-bin has some bad internal properties?

inotifywait and sed is in the middle of patching the consts.rs file, so you get
half of a source-file:

error: this file contains an un-closed delimiter
   --> /build/rust/src/rustc-1.38.0-src/build/i686-unknown-linux-gnu/stage2-tools/i686-unknown-linux-gnu/release/build/typenum-483ffd21aece8552/out/consts.rs:266:56

Maybe some mandatory file locking helps?

Otherwise, I start considering a "fuse-patching" pseudo-filesystem.

flock doesn't help, the file is now locked, but rust sees it as a sequence of \0.

The following works: wait for inotify event close_write on file consts.rs and patch
it then.
Kill the watcher (which also kills inotifywait).

Now I can rebuild rust 1.38 from 1.37, but I still have problems rebuilding 1.38
with an 1.38 compiler (*sigh*).

As we have to assume that rust 1.37 and 1.36 are broken, let's bootstrap again a
rust 1.38 from rust-bin 1.37.

Now someting hangs in the middle..

upstream fixed the boostrapping 1.38 with 1.38 issue (seems to be just some
superflous unsafe directives):

  # Fix bootstrap to compile with 1.38
  patch -Np1 -i ../bootstrap-1.38.patch

Trying again to rebuild 1.38 with 1.38 (both for i686 and pentium4) against my bootstrapped
1.38 version. When this works, I'll build official version in the regular way..

  cat > config.toml <<EOF
...
EOF

This seems to be a new trend. I don't like it, as it makes patching really hard..

i686 1.38 rebuild of 1.38 results in:

error: Could not compile `typenum`.
warning: build failed, waiting for other jobs to finish...
error: build failed
command did not execute successfully: "/usr/bin/cargo" "build" "--target" "i686-unknown-linux-gnu" "-j" "16" "--release" "--frozen" "--manifest-path" "/build/rust/src/rustc-1.38.0-src/src/tools/cargo/Cargo.toml" "--message-format" "json"
expected success, got: exit code: 101
failed to run: /build/rust/src/rustc-1.38.0-src/build/bootstrap/debug/bootstrap install -j16
Build completed unsuccessfully in 0:38:12

no error messages, exit code 101 means what exactly?