The home lab is my LAVA development environment and I've recently got two iMX53 Quick Start Boards working with a typical LAVA master image based on a Linaro build of Linux 3.1.0 for mx5 with a Ubuntu Oneiric Ocelot 11.10 rootfs:
3.1.0-1002-linaro-lt-mx5 (buildd@hubbard) (gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3) ) #13-Ubuntu PREEMPT Fri Dec 16 01:21:07 UTC 2011
As part of my Debian work, it was clearly time to look at a current, Debian, kernel and rootfs and as I'm developing and testing on Debian unstable, this would necessarily mean testing the Debian ARMMP (multi-platform) kernel which replaces the mx5 build used in Wheezy:
Linux version 3.12-1-armmp (debian-kernel@lists.debian.org) (gcc version 4.8.2 (Debian 4.8.2-10) ) #1 SMP Debian 3.12.9-1 (2014-02-01)
I will be scripting the creation of a suitable image for these tests and there are other changes planned in LAVA to make it easier to build suitable images, but it is useful to document just how I worked out how to build the first image.
Manual build steps
First, I've already discovered that the u-boot on the iMX53 doesn't like ext4, so the first step was to prepare an ext3 filesystem for the image. With a SATA drive attached, it was also much better to use that than the SD card, at least for generating the image. I'm also doing this natively, so I am working inside the booted master image. This is fine as the master is designed to manipulate test images, so the only package I needed to install on the LAVA master image was debootstrap I had an empty SATA drive to play with for these tests, so first prepare an ext3 filesystem:
# mkfs.ext3 /dev/sda1 # mkdir /mnt/sata # mount /dev/sda1 /mnt/sata # mkdir /mnt/sata/chroots/
Start the debootstrap:
# apt-get update # apt-get install debootstrap # debootstrap --arch=armhf --include=linux-image-armmp \ --verbose unstable \ /mnt/sata/chroots/unstable-armhf http://ftp.uk.debian.org/debian
Various actions in this chroot will need proc, so mount it here:
# chroot /mnt/sata/chroots/unstable-armhf # mount proc -t proc /proc # mount devpts -t devpts /dev/pts # exit
You may also have to edit the apt sources - the LAVA master image doesn't have editors installed, so either use echo or download an edited file. I'm using:
deb http://ftp.uk.debian.org/debian sid main
flash-kernel needs changes
For the initial tests, I've got to get this image to boot directly from u-boot, so flash-kernel is going to be needed inside the chroot and to get the iMX53 to work with the ARMMP kernel and Device Tree, flash-kernel will need an update which will mean a patch:
# chroot /mnt/sata/chroots/unstable-armhf # apt-get update # apt-get install patch flash-kernel # cp /usr/share/flash-kernel/db/all.db /home # cd /home # patch -p2
The patch itself goes through a couple of iterations. Initially, it is enough to use:
Machine: Freescale MX53 LOCO Board -Kernel-Flavors: mx5 +Kernel-Flavors: armmp +DTB-Id: imx53-qsb.dtb +DTB-Append: yes
Later, once the device has booted with the ARMMP kernel, the Machine Id can be updated to distinguish it from the mx5 flavour (from /proc/cpuinfo) and use the model name from the Device Tree (/proc/device-tree/model):
diff --git a/db/all.db b/db/all.db index fab3407..41f6c78 100644 --- a/db/all.db +++ b/db/all.db @@ -62,6 +62,18 @@ U-Boot-Initrd-Address: 0x00800000 Required-Packages: u-boot-tools Bootloader-Sets-Incorrect-Root: yes +Machine: Freescale i.MX53 Quick Start Board +Kernel-Flavors: armmp +DTB-Id: imx53-qsb.dtb +DTB-Append-From: 3.12 +Boot-DTB-Path: /boot/dtb +U-Boot-Kernel-Address: 0x70008000 +U-Boot-Initrd-Address: 0x0 +Boot-Kernel-Path: /boot/uImage +Boot-Initrd-Path: /boot/uInitrd +Required-Packages: u-boot-tools +Bootloader-Sets-Incorrect-Root: no + Machine: Freescale MX53 LOCO Board Kernel-Flavors: mx5 U-Boot-Kernel-Address: 0x70008000
(I will be filing this patch in a bug report against flash-kernel soon.) With that patched, update and run flash-kernel:
# mv all.db /usr/share/flash-kernel/db/all.db # flash-kernel flash-kernel: installing version 3.12-1-armmp Generating kernel u-boot image... done. Taking backup of uImage. Installing new uImage. Generating initramfs u-boot image... done. Taking backup of uInitrd. Installing new uInitrd. Installing new dtb. # exit
LAVA overlays
This will be a LAVA test image and it needs an overlay - if you want a vanilla image, set up a passwd inside the chroot instead:
# cd /mnt/sata/chroots/unstable-armhf/ # wget --no-check-certificate \ https://launchpad.net/~linaro-maintainers/+archive/overlay/+files/linaro-overlay-minimal_1112.2_all.deb # wget --no-check-certificate \ https://launchpad.net/~linaro-maintainers/+archive/overlay/+files/linaro-overlay_1112.2_all.deb # chroot /mnt/sata/chroots/unstable-armhf/ # dpkg -i linaro-overlay-minimal_1112.2_all.deb linaro-overlay_1112.2_all.deb # rm linaro-overlay-minimal_1112.2_all.deb linaro-overlay_1112.2_all.deb # exit
Changes to allow the chroot to boot
Now the chroot needs setting up as a boot image:
# chroot /mnt/sata/chroots/unstable-armhf/ # echo T0:23:respawn:/sbin/getty -L ttymxc0 115200 vt102 >> ./etc/inittab # echo auto lo eth0 > ./etc/network/interfaces.d/mx53loco # echo iface lo inet loopback >> ./etc/network/interfaces.d/mx53loco # echo iface eth0 inet dhcp >> ./etc/network/interfaces.d/mx53loco # apt-get clean # umount /dev/pts # umount /proc # exit
Partitioning as a LAVA test image
This would be enough as a single partition test image but, currently, LAVA expects a much more hard-wired image. Depending on the history of the device and the need for LAVA to be able to put fresh kernel builds together with a known rootfs, LAVA has used a separate /boot and / partition in the test image for nearly all boards. Standard LAVA test images for many boards (like the iMX53) also have a small unallocated space at the start of the SD card, so until I can get LAVA upstream to handle test images of arbitrary design, I'm adapting the image to suit the expectations inside LAVA. Yes, I know but it's better to get something working before spending time fixing things to make it work better. It will be fixed, in time.
So I needed to separate out the /boot contents from the rest of the rootfs - whilst keeping the chroot itself in a state which I can easily update and use to create other images:
# cd /mnt/sata/chroots/unstable-armhf/boot/ # tar -czf ../../boot.tar.gz ./* # cd .. # tar -czf ../root.tar.gz ./*
Now the test image file and its partitions:
# dd if=/dev/zero of=/mnt/sata/images/empty/debian.img bs=1M count=1024 # cp /mnt/sata/images/empty/debian.img /mnt/sata/images/debian-unstable-armhf-armmp.img # losetup /dev/loop0 /mnt/sata/images/debian-unstable-armhf-armmp.img # parted /dev/loop0 -s unit mb mktable msdos # parted /dev/loop0 -s unit mb mkpart primary 1 10 # parted /dev/loop0 -s unit mb mkpart primary 11 110 # parted /dev/loop0 -s unit mb mkpart primary 111 1024
I did make the mistake of using kpartx at this stage but there are many areas of confusion when translating the kpartx output to offsets for mount when parted is easier:
# parted /dev/loop0 unit B -s print Model: (file) Disk /dev/loop0: 1073741824B Sector size (logical/physical): 512B/512B Partition Table: msdos Number Start End Size Type File system Flags 1 1048576B 10485759B 9437184B primary 2 10485760B 110100479B 99614720B primary 3 110100480B 1024458751B 914358272B primary
Use the Start numbers and use losetup to create the loop devices for each partition:
# losetup -o 10485760 /dev/loop1 /dev/loop0 # losetup -o 110100480 /dev/loop2 /dev/loop0 # mkfs.vfat /dev/loop1 # mkfs.ext3 /dev/loop2
Now clean up the loop mountpoints:
# losetup -d /dev/loop2 # losetup -d /dev/loop1 # losetup -d /dev/loop0 # losetup -a
losetup -a should return nothing. If it doesn't, investigate the contents of /dev/mapper and use dmsetup remove until losetup -a does report empty. Otherwise the subsequent stages will fail. Now deploy the /boot contents into the empty image:
# mount -oloop,offset=10485760 /mnt/sata/images/debian-unstable-armhf-armmp.img /mnt/boot/ # pushd /mnt/boot/ # tar -xzf /mnt/sata/chroots/boot.tar.gz # popd # sync # umount /mnt/boot/
and the / contents (removing the duplicate ./boot contents):
# mount -oloop,offset=110100480 /mnt/sata/images/debian-unstable-armhf-armmp.img /mnt/root/ # pushd /mnt/root # tar -xzf /mnt/sata/chroots/root.tar.gz # rm ./boot/* # popd # sync # umount /mnt/root
Check the image
# mount -oloop,offset=10485760 debian-unstable-armhf-armmp.img /mnt/boot # ls /mnt/boot config-3.12-1-armmp initrd.img-3.12-1-armmp System.map-3.12-1-armmp uImage uInitrd vmlinuz-3.12-1-armmp # umount /mnt/boot # mount -oloop,offset=110100480 debian-unstable-armhf-armmp.img /mnt/root # ls /mnt/root bin boot dev etc home initrd.img lib lost+found media mnt opt proc root run sbin srv sys tmp usr var vmlinuz # ls /mnt/root/bin/auto-serial-console /mnt/root/bin/auto-serial-console # umount /mnt/root # md5sum debian-unstable-armhf-armmp.img
Downloads
Yes, I have put that file online, if you are interested. Do read the readme first though.
Getting the image off the device
# ip a # python -m SimpleHTTPServer 80 2>/dev/null
then wget (just http://, IP address / and the filename), md5sum - finish with Ctrl-C. Depending on setup, it may be quicker to transfer the uncompressed file over a LAN than to let the device compress it. Your main machine will do the compression much faster, even with a larger download. (It also helps to not compress the image on the device, you can test the mount offsets more easily - and do check that the image can be mounted with the offsets from parted.)
Results
- complete log of the first successful LAVA test of a Debian ARMMP kernel on an iMX53 (153k)
- LAVA submission JSON (only usable with a local URL of the image and a few tweaks to the standard LAVA mx53loco device configuration)
- LAVA result bundle (JSON) (75k)
# cat /proc/cpuinfo processor : 0 model name : ARMv7 Processor rev 5 (v7l) Features : swp half thumb fastmult vfp edsp thumbee neon vfpv3 tls vfpd32 CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x2 CPU part : 0xc08 CPU revision : 5 Hardware : Freescale i.MX53 (Device Tree Support) Revision : 0000 Serial : 0000000000000000
# uname -a Linux imx53-02 3.12-1-armmp #1 SMP Debian 3.12.9-1 (2014-02-01) armv7l GNU/Linux
# lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux unstable (sid) Release: unstable Codename: sid
Next!
Yes, there is a lot to do from here on.
- The creation of the test image needs to be smoothed out and scripted. I'm thinking that this would go into vmdebootstrap. Lars?
- LAVA needs to be less presumptive about the contents and partitioning of the test image.
- the Debian ARMMP kernel itself needs to regain SATA support (an update is already pending).
- I've also got two Cubieboard2 devices and there is a sun7i-a20-cubieboard2.dtb file in the ARMMP kernel which deserves some investigation.
Then there is the whole issue of actually making this multi-platform. After all, it is far from ideal that a multi-platform kernel package has to have platform-specific operations using flash-kernel at each update. So Grub on ARM is going to be on the list of things to investigate.