Re: HP mv2120 support in Orion tree

[Marc Singer]

On Tue, Jun 10, 2008 at 08:23:25PM +0200, Martin Michlmayr wrote:
> Eugene, let me introduce you to Marc Singer.  He recently started
> working on Debian support for the HP mv2120.
>
> Marc, Eugene is interested in Debian support too.  Eugene seems to
> know a lot about the boot process.
>
> My answers are inline below (but maybe we should move this discussion
> to debian-arm for all to see?):

Good plan.

>
> * Eugene San <eugenesan@...> [2008-06-02 08:15]:
> > Hi,
> >
> > Few questions first:
> > 1) How exactly install images created in Debian? Is there any special  
> > automatic procedure or they just created and uploaded by maintainer?
>
> There's a project called "debian-installer" which puts all the
> installer components together.  It also generates the actual installer
> image.
>
> > 2) Runtime initrd and kernel, is their build and usage procedure is  
> > forced by some kind policy or/and automatic procedure?
> >    Can we change that? Let's say, run that machine without initrd and use
> > uImage instead of standard image?
>
> The initrd is generated by initramfs-tools.  We can change some
> things... but there are some limits to what we can change because
> other machines have different requirements, and we have to use one
> kernel on all Orion based devices.

I believe we're going to have to generate a uImage to get the system
to boot without replacing uBoot in flash.  Sure.  We should be able to
use the same kernel on all machines as long as we can perform a
post-install hook to cope with the initramfs binding, or whatever.

> > 3) Can we inject mv21xx/51xx support now or will have to wait for 2.6.27  
> > to land in SID?
>
> Yes.  In fact, we have Orion support in Debian sid already.  2.6.25 in
> sid has support for the mv2120 included.  At the moment, the mv2120
> patch is not included in the Debian 2.6.26 patch... but I can include
> it once someone backports the 2.6.27 patch to 2.6.26.
>
> > 4) Is install procedure of that kind of devices is automatic to avoid use
> > of RS232? How set of packages and partitioning is controlled?
>
> Right, to avoid the need for a serial install.  The installer includes
> a partitioner, so you can choose how to partition it... you can also
> choose which packages to install.

I've been working on a tool to leverage the RCVR recovery function in
uBoot to bootstrap the install process.  This will allow us to run the
installer without opening the box.  I am catching the beacon and
starting the TFTP transfer.  I should have something working in a few
days that can upload an image.

> > I don't think initrd can be part of regular boot without uBoot  
> > environment modifications, but I think there is userspace utility that  
> > can help.
>
> Marc has some ideas on this.

  1) Use a multipart uImage to load kernel and ramdisk.  I don't know
     if uBoot supports this, but it is in their format.  This requires
     construction of a custom uImage in the kernel install post-hook.
  2) Use a singlepart uImage with an initramfs appended to the
     kernel.  This requires that we prepare this special kernel as
     part of building the kernel install post-hook.
  3) Use either a kernel shim or APEX to perform fixups.  This would
     be prepended to the kernel image and run automatically and is yet
     another special step on kernel image install.
  4) Last, do a complete port of APEX that can read from the hard
     drive and use APEX to load the kernel and ramdisk from the hard
     drive.  This requires the most new work, so I'm hoping that one
     of the other options works and we don't have to take this tack.

> >> Can you describe the recovery procedure?  Do you know how it works?
> >> Because I don't think I know exactly how it works.
> >>
> >>  
> > First, I have some knowledge of the recovery procedure. If you need to  
> > know more, just ask ;-)
> > After recovery mode initialized, by holding front reset while powering up
> > or invoking "rcv" in uBoot:
> > 1) uBoot starts dhcp client. If that fails it selects defined in uBoot  
> > recovery IP.
> > 2) Waits for client to authorize and then waits for TFTP upload from  
> > clientside. First 2M reserved for kernel uImage, the rest is uInitrd
> > image.
> > 3) uBoot has it's own treatment for combined images. uBoot unpacks both  
> > images into RAM and initiate regular linux+initrd boot.
> >    Tool called mkimage, which is part of uBoot source can be easily  
> > disintegrated into 3 separate source files, is used to create such images.
> >    uBoot just wraps regular zImage and zInitrd with special header.
> > 4) We can use original recover tool, but it exists only for windows.
>
> I think Marc will find this info useful.
>
> Eugene, do you know howto produce such a rescue image?

This matches what I've found so far.  Thanks.

One thing that is really odd, though.  I found yesterday that holding
the reset button while powering the device on is no longer running the
recovery procedure automatically.  I cannot explain why.  I may need
to perform a complete recovery of the system to put it back into the
factory default setup.

Eugene, have you ever seen this behavior?

Cheers.


--
To UNSUBSCRIBE, email to debian-arm-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Marc Singer]

On Tue, Jun 10, 2008 at 10:45:37PM +0300, Eugene San wrote:

> Martin: Thank you for answering my question.
>
> To create install image we need uImage, uInitrd and mkimage
> a) uImage can be provided by make uImage in kernel source tree or by  
> converting regular image with: |mkimage -A arm -O linux -T kernel -C  
> none -a 0x80000 -e 0x80000 -n "Linux 2.6" -d linux.bin uImage.bin |
>    ( http://www.denx.de/wiki/view/DULG/LinuxConfiguration#Section_6.2 )
> b) uInitr can be created from regular inird by: "mkimage -T ramdisk -C  
> none -n 'Test Ramdisk Image -d ramdisk.img.gz uRamdisk"
>    ( http://www.denx.de/wiki/view/DULG/RootFileSystemOnARamdisk )
> c) mkimage is available as package in SID,  
> http://packages.debian.org/search?keywords=uboot-mkimage
> Final combined image (recovery.img), that may be uploaded and executed  
> on device using original recovery utility, is just two files glued
> together.
> uImage must be padded to 2M and uInitrd to 8M and maybe compressed by  
> replacing "-C none" with "-C gzip".
>
> If you need more help, I'll be glad to assist.

Are you sure the ramdisk must be padded to 8MiB?  Is the full image
size supposed to be 10MiB, 2 + 8?


--
To UNSUBSCRIBE, email to debian-arm-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Eugene San]

Marc Singer wrote:

On Tue, Jun 10, 2008 at 10:45:37PM +0300, Eugene San wrote:
  

Martin: Thank you for answering my question.

To create install image we need uImage, uInitrd and mkimage
a) uImage can be provided by make uImage in kernel source tree or by  
converting regular image with: |mkimage -A arm -O linux -T kernel -C  
none -a 0x80000 -e 0x80000 -n "Linux 2.6" -d linux.bin uImage.bin |
   ( http://www.denx.de/wiki/view/DULG/LinuxConfiguration#Section_6.2 )
b) uInitr can be created from regular inird by: "mkimage -T ramdisk -C  
none -n 'Test Ramdisk Image -d ramdisk.img.gz uRamdisk"
   ( http://www.denx.de/wiki/view/DULG/RootFileSystemOnARamdisk )
c) mkimage is available as package in SID,  
http://packages.debian.org/search?keywords=uboot-mkimage
Final combined image (recovery.img), that may be uploaded and executed  
on device using original recovery utility, is just two files glued 
together.
uImage must be padded to 2M and uInitrd to 8M and maybe compressed by  
replacing "-C none" with "-C gzip".

If you need more help, I'll be glad to assist.
    

Are you sure the ramdisk must be padded to 8MiB?  Is the full image
size supposed to be 10MiB, 2 + 8?

  

uImage must be at 0x0 and uInitrd must be at 0x200000 since uBoot expects them to be there.
uInitrd size is limited to 8Mb in original first stage image, probably by in Kernel ramdisk size. Bigger image may work, need to be tested.
In case that will not work for us, we still can use second stage image, which maybe much bigger.

[Marc Singer]

On Wed, Jun 11, 2008 at 12:55:25AM +0300, Eugene San wrote:

> Marc Singer wrote:
>
>> On Tue, Jun 10, 2008 at 10:45:37PM +0300, Eugene San wrote:
>>  
>>> Martin: Thank you for answering my question.
>>>
>>> To create install image we need uImage, uInitrd and mkimage
>>> a) uImage can be provided by make uImage in kernel source tree or by  
>>> converting regular image with: |mkimage -A arm -O linux -T kernel -C  
>>> none -a 0x80000 -e 0x80000 -n "Linux 2.6" -d linux.bin uImage.bin |
>>>    ( http://www.denx.de/wiki/view/DULG/LinuxConfiguration#Section_6.2 )
>>> b) uInitr can be created from regular inird by: "mkimage -T ramdisk
>>> -C  none -n 'Test Ramdisk Image -d ramdisk.img.gz uRamdisk"
>>>    ( http://www.denx.de/wiki/view/DULG/RootFileSystemOnARamdisk )
>>> c) mkimage is available as package in SID,  
>>> http://packages.debian.org/search?keywords=uboot-mkimage
>>> Final combined image (recovery.img), that may be uploaded and
>>> executed  on device using original recovery utility, is just two
>>> files glued together.
>>> uImage must be padded to 2M and uInitrd to 8M and maybe compressed by
>>>  replacing "-C none" with "-C gzip".
>>>
>>> If you need more help, I'll be glad to assist.
>>>    
>>
>> Are you sure the ramdisk must be padded to 8MiB?  Is the full image
>> size supposed to be 10MiB, 2 + 8?
>>
>>  
> uImage must be at 0x0 and uInitrd must be at 0x200000 since uBoot  
> expects them to be there.
> uInitrd size is limited to 8Mb in original first stage image, probably  
> by in Kernel ramdisk size. Bigger image may work, need to be tested.
> In case that will not work for us, we still can use second stage image,  
> which maybe much bigger.

Good.

I'll get back to you all when I can run some tests.

BTW, do you know if uBoot will accept a multi-part image via the
bootext2 command?


--
To UNSUBSCRIBE, email to debian-arm-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Eugene San]

Marc Singer wrote:

On Tue, Jun 10, 2008 at 08:23:25PM +0200, Martin Michlmayr wrote:
  

Eugene, let me introduce you to Marc Singer.  He recently started
working on Debian support for the HP mv2120.

Marc, Eugene is interested in Debian support too.  Eugene seems to
know a lot about the boot process.

My answers are inline below (but maybe we should move this discussion
to debian-arm for all to see?):
    

Good plan.

  

* Eugene San eugenesan@... [2008-06-02 08:15]:
    

Hi,

Few questions first:
1) How exactly install images created in Debian? Is there any special  
automatic procedure or they just created and uploaded by maintainer?
      

There's a project called "debian-installer" which puts all the
installer components together.  It also generates the actual installer
image.

    

2) Runtime initrd and kernel, is their build and usage procedure is  
forced by some kind policy or/and automatic procedure?
   Can we change that? Let's say, run that machine without initrd and use 
uImage instead of standard image?
      

The initrd is generated by initramfs-tools.  We can change some
things... but there are some limits to what we can change because
other machines have different requirements, and we have to use one
kernel on all Orion based devices.
    

I believe we're going to have to generate a uImage to get the system
to boot without replacing uBoot in flash.  Sure.  We should be able to
use the same kernel on all machines as long as we can perform a
post-install hook to cope with the initramfs binding, or whatever.
  

Do you think of single initrd or several for each machine?
What do you expect size of compressed initrd,  to be?

3) Can we inject mv21xx/51xx support now or will have to wait for 2.6.27  
to land in SID?
      

Yes.  In fact, we have Orion support in Debian sid already.  2.6.25 in
sid has support for the mv2120 included.  At the moment, the mv2120
patch is not included in the Debian 2.6.26 patch... but I can include
it once someone backports the 2.6.27 patch to 2.6.26.

    

4) Is install procedure of that kind of devices is automatic to avoid use 
of RS232? How set of packages and partitioning is controlled?
      

Right, to avoid the need for a serial install.  The installer includes
a partitioner, so you can choose how to partition it... you can also
choose which packages to install.
    

I've been working on a tool to leverage the RCVR recovery function in
uBoot to bootstrap the install process.  This will allow us to run the
installer without opening the box.  I am catching the beacon and
starting the TFTP transfer.  I should have something working in a few
days that can upload an image.
  

Good, if you still need help contact me.

I don't think initrd can be part of regular boot without uBoot  
environment modifications, but I think there is userspace utility that  
can help.
      

Marc has some ideas on this.
    

  1) Use a multipart uImage to load kernel and ramdisk.  I don't know
     if uBoot supports this, but it is in their format.  This requires
     construction of a custom uImage in the kernel install post-hook.
  2) Use a singlepart uImage with an initramfs appended to the
     kernel.  This requires that we prepare this special kernel as
     part of building the kernel install post-hook.
  3) Use either a kernel shim or APEX to perform fixups.  This would
     be prepended to the kernel image and run automatically and is yet
     another special step on kernel image install. 
  4) Last, do a complete port of APEX that can read from the hard
     drive and use APEX to load the kernel and ramdisk from the hard
     drive.  This requires the most new work, so I'm hoping that one
     of the other options works and we don't have to take this tack.

  

Can you describe the recovery procedure?  Do you know how it works?
Because I don't think I know exactly how it works.

  
        

First, I have some knowledge of the recovery procedure. If you need to  
know more, just ask ;-)
After recovery mode initialized, by holding front reset while powering up 
or invoking "rcv" in uBoot:
1) uBoot starts dhcp client. If that fails it selects defined in uBoot  
recovery IP.
2) Waits for client to authorize and then waits for TFTP upload from  
clientside. First 2M reserved for kernel uImage, the rest is uInitrd 
image.
3) uBoot has it's own treatment for combined images. uBoot unpacks both  
images into RAM and initiate regular linux+initrd boot.
   Tool called mkimage, which is part of uBoot source can be easily  
disintegrated into 3 separate source files, is used to create such images.
   uBoot just wraps regular zImage and zInitrd with special header.
4) We can use original recover tool, but it exists only for windows.
      

I think Marc will find this info useful.

Eugene, do you know howto produce such a rescue image?
    

This matches what I've found so far.  Thanks.

One thing that is really odd, though.  I found yesterday that holding
the reset button while powering the device on is no longer running the
recovery procedure automatically.  I cannot explain why.  I may need
to perform a complete recovery of the system to put it back into the
factory default setup.

Eugene, have you ever seen this behavior?
  

No, that looks strange for me too. Probably one of default parameters got vanished.
You may invoke "rcvr" start recovery procedure or alternately invoke "resetenv; reset;" to restore uBoot defaults.
Hope "resetenv" is implemented in your version of uBoot.

Cheers.

  

[Eugene San]

Marc Singer wrote:

On Wed, Jun 11, 2008 at 12:55:25AM +0300, Eugene San wrote:
  

Marc Singer wrote:

    

On Tue, Jun 10, 2008 at 10:45:37PM +0300, Eugene San wrote:
  
      

Martin: Thank you for answering my question.

To create install image we need uImage, uInitrd and mkimage
a) uImage can be provided by make uImage in kernel source tree or by  
converting regular image with: |mkimage -A arm -O linux -T kernel -C  
none -a 0x80000 -e 0x80000 -n "Linux 2.6" -d linux.bin uImage.bin |
   ( http://www.denx.de/wiki/view/DULG/LinuxConfiguration#Section_6.2 )
b) uInitr can be created from regular inird by: "mkimage -T ramdisk 
-C  none -n 'Test Ramdisk Image -d ramdisk.img.gz uRamdisk"
   ( http://www.denx.de/wiki/view/DULG/RootFileSystemOnARamdisk )
c) mkimage is available as package in SID,   
http://packages.debian.org/search?keywords=uboot-mkimage
Final combined image (recovery.img), that may be uploaded and 
executed  on device using original recovery utility, is just two 
files glued together.
uImage must be padded to 2M and uInitrd to 8M and maybe compressed by 
 replacing "-C none" with "-C gzip".

If you need more help, I'll be glad to assist.
    
        

Are you sure the ramdisk must be padded to 8MiB?  Is the full image
size supposed to be 10MiB, 2 + 8?

  
      

uImage must be at 0x0 and uInitrd must be at 0x200000 since uBoot  
expects them to be there.
uInitrd size is limited to 8Mb in original first stage image, probably  
by in Kernel ramdisk size. Bigger image may work, need to be tested.
In case that will not work for us, we still can use second stage image,  
which maybe much bigger.
    

Good.

I'll get back to you all when I can run some tests.

BTW, do you know if uBoot will accept a multi-part image via the
bootext2 command?

  

What do you mean by multipart? Glued linux+initrd?
bootm can accept 2 parameters first for uImage and second for uInitrd.
You may use e2fsload to get uInitrd and uImage from disk and the use bootm to start.

[Marc Singer]

On Wed, Jun 11, 2008 at 01:41:34AM +0300, Eugene San wrote:
> What do you mean by multipart? Glued linux+initrd?
> bootm can accept 2 parameters first for uImage and second for uInitrd.
> You may use e2fsload to get uInitrd and uImage from disk and the use  
> bootm to start.

There is a uImage type called multi.  It's type number 4.  It can
store more than one sub-image in a single wrapper.




--
To UNSUBSCRIBE, email to debian-arm-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Marc Singer]

On Wed, Jun 11, 2008 at 01:18:35AM +0300, Eugene San wrote:
>> I believe we're going to have to generate a uImage to get the system
>> to boot without replacing uBoot in flash.  Sure.  We should be able to
>> use the same kernel on all machines as long as we can perform a
>> post-install hook to cope with the initramfs binding, or whatever.
>>  
> Do you think of single initrd or several for each machine?
> What do you expect size of compressed initrd,  to be?

I don't really know since I'm not going to build them myself.  I
believe that the Debian installer starts with a single ramdisk
rootfs.  It loads compoenents from the network and installs the same
way.

The initrd's for the current Debian release are about 3.5MiB.  The
Debian installer images for the NSLU2 are about 5 MiB, IIRC.

>>
>> Eugene, have you ever seen this behavior?
>>  
> No, that looks strange for me too. Probably one of default parameters  
> got vanished.
> You may invoke "rcvr" start recovery procedure or alternately invoke  
> "resetenv; reset;" to restore uBoot defaults.
> Hope "resetenv" is implemented in your version of uBoot.

Good tip.  I'll try it.


--
To UNSUBSCRIBE, email to debian-arm-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Marc Singer]

On Wed, Jun 11, 2008 at 01:18:35AM +0300, Eugene San wrote:
>> Eugene, have you ever seen this behavior?
>>  
> No, that looks strange for me too. Probably one of default parameters  
> got vanished.
> You may invoke "rcvr" start recovery procedure or alternately invoke  
> "resetenv; reset;" to restore uBoot defaults.
> Hope "resetenv" is implemented in your version of uBoot.

So, there is such a command.  Using it erases my serial number and MAC
address.  Fortunately, I saved beforehand.

I still see this odd behavior.  If I put the device into the recovery
mode (hold reset and power-on) and then detach the power supply, it is
likely to not start the recover mode next time.

Here is the before and after.  Mostly benign, but I haven't look into
what all of these mean.

  1c1
  < BOOT_MODE=normal
  ---
  > BOOT_MODE=runin
  8,9c8
  < bootargs_root=root=/dev/nfs rw console=ttyS0,115200
  < bootcmd=bootext2 0,1:1,2 0x400000 /boot/uImage /dev/sda /dev/sdb
  ---
  > bootcmd=bootext2 0,1:1,2 $(loadAddr) /boot/$(image_name) /dev/sda /dev/sdb;
  22c21
  < ethaddr=00:0a:e4:83:a5:71
  ---
  > ethaddr=00:00:00:00:51:81
  29d27
  < mfgtest_state=final_tested_ok
  35,36c33,34
  < runintime=13000
  < serialNo=5561n01d01g72800061j0sb
  ---
  > runintime=14400
  > serialNo=AS100352999999
  44c42
  < Environment size: 1041/4092 bytes
  ---
  > Environment size: 960/4092 bytes


--
To UNSUBSCRIBE, email to debian-arm-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Martin Michlmayr]

* Marc Singer <elf@...> [2008-06-10 12:00]:
> > My answers are inline below (but maybe we should move this discussion
> > to debian-arm for all to see?):
> Good plan.

Just to provide some background: we're discussing some things that are
needed to add Debian support to HP's mv2120 NAS device.

As far as I can see, the following items are needed to add support:we need to find solutions for the following

 - make a mv2120 patch for 2.6.26, i.e. backport the patch from
   Marvell's git tree to plain 2.6.26.  I asked Sylver Bruneau
   whether he can help with this.

 - figure out how the rescue mode works and how to construct a rescue
   image that the device can boot: it seems Eugene San has figured out
   most or all of this already.

 - write a server for Linux that can serve these rescue images: I
   think Marc Singer was looking into this (?).

 - figure out the best way to make one image that contains the kernel and
   ramdisk: Marc had some ideas about this.

 - add a check to partman that ensures that sda1 is an ext2/ext3 partition
   that is mounted on either /boot or /.  This one is easy.

--
Martin Michlmayr
http://www.cyrius.com/


--
To UNSUBSCRIBE, email to debian-arm-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Marc Singer]

On Sun, Jun 15, 2008 at 12:18:43PM +0200, Martin Michlmayr wrote:

> * Marc Singer <elf@...> [2008-06-10 12:00]:
> > > My answers are inline below (but maybe we should move this discussion
> > > to debian-arm for all to see?):
> > Good plan.
>
> Just to provide some background: we're discussing some things that are
> needed to add Debian support to HP's mv2120 NAS device.
>
>  - figure out how the rescue mode works and how to construct a rescue
>    image that the device can boot: it seems Eugene San has figured out
>    most or all of this already.
>
>  - write a server for Linux that can serve these rescue images: I
>    think Marc Singer was looking into this (?).

I've figured out these last two; I have a program that serves the
images.  One hitch I'm encountering is that I'm unable to enter the
rescue mode by holding the rest button...any more.  It used to work.
I'm going to do a little hardware debugging to see if there is
something wrong with the button.  If I'm not the only person with this
problem, we have have to figure out an alternative.


--
To UNSUBSCRIBE, email to debian-arm-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Marc Singer]

On Sun, Jun 15, 2008 at 12:18:43PM +0200, Martin Michlmayr wrote:
>  - figure out the best way to make one image that contains the kernel and
>    ramdisk: Marc had some ideas about this.

I've done a quick refresh on initramfs.  It looks like the only way to
get the kernel to auto-load the rootfs without passing initrd_start
address is to link the initramfs into the kernel.  So, either

 1) we relink the kernel image with an initramfs image--may not be too
    terrible--which means we wouldn't have to do anything to the boot
    process or
 2) we see if we can get uboot multi-images to do the trick. or
 3) we do the normal thing of appending an initrd (of some sort) to
    the kernel, wrap the whole kit into a single uImage, and use a
    kernel shim to fixup the kernel arguments to include ATAGS for the
    ramdisk image.

The real trouble with 1 is that we need the ELF image of the kernel to
do the relink.  I don't see this being available and it looks like the
initramfs is linked into the middle of the kernel, so some shenanigans
with the zImage and the link map are probably out of the question.

It looks like there is multi-image support in the uboot as shipped.
We'll have to test it.  Martin, how could I generate such a rootfs?  I
don't really care if the FS starts the installer.  I just want to see
that the kernel can mount it.


--
To UNSUBSCRIBE, email to debian-arm-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Martin Michlmayr]

* Marc Singer <elf@...> [2008-06-17 11:46]:
> It looks like there is multi-image support in the uboot as shipped.
> We'll have to test it.  Martin, how could I generate such a rootfs?  I
> don't really care if the FS starts the installer.  I just want to see
> that the kernel can mount it.

If you just need a random d-i ramdisk for testing, you can grab
http://merkel.debian.org/~tbm/tmp/orion/qnap/ts-209/initrd.gz

--
Martin Michlmayr
http://www.cyrius.com/


--
To UNSUBSCRIBE, email to debian-arm-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...