Francis Luong

View Original

#JUNOS - Recovering from Alternate Media (YMMV)

Had a situation at work where I had to remind myself of something so that means it’s time for a new blog post.

--- JUNOS 12.3R3.4 built 2013-06-14 00:09:12 UTC
---
--- NOTICE: System is running on alternate media device      (/dev/ad1s1a).
---

fluong@tickle-me-elmo-re0> show system storage | no-more 
Filesystem              Size       Used      Avail  Capacity   Mounted on
/dev/ad1s1a             3.5G       283M       3.1G        8%  / <<<<<
devfs                   1.0K       1.0K         0B      100%  /dev
/dev/md0                 41M        41M         0B      100%  /packages/mnt/jbase
/dev/md1                 32M        32M         0B      100%  /packages/mnt/jkernel64-12.1R1.9
/dev/md2                 73M        73M         0B      100%  /packages/mnt/jpfe-X960-12.1R1.9
/dev/md3                5.0M       5.0M         0B      100%  /packages/mnt/jdocs-12.1R1.9
/dev/md4                 78M        78M         0B      100%  /packages/mnt/jroute-12.1R1.9
/dev/md5                 28M        28M         0B      100%  /packages/mnt/jcrypto64-12.1R1.9
/dev/md6                 46M        46M         0B      100%  /packages/mnt/jpfe-common-12.1R1.9
/dev/md7                388M       388M         0B      100%  /packages/mnt/jruntime-12.1R1.9
/dev/md8                7.9G        22K       7.2G        0%  /tmp
/dev/md9                7.9G        15M       7.2G        0%  /mfs
/dev/ad1s1e             394M        42K       390M        0%  /config
procfs                  4.0K       4.0K         0B      100%  /proc
/dev/ad1s1f              18G       2.3G        14G       14%  /var

Here’s the initial scenario. Routing-Engine re0 is booting from alternate media. IN MOST CASES this means a compact-flash on the routing has gone bad and has to be replaced by RMA, but in this case I happen to know that it’s a new RE and we had a USB install that went south. Keep this in mind and know that your mileage may vary with this one.

Other interesting considerations for this scenario is that for this router, a remote hands technician is not on site so we don’t have cheap easy options to do another USB install. Luckily JUNOS provides a means to rewrite the image on the compact-flash if you’re able to boot off the HDD/SSD: “request system snapshot partition

{backup}
root@tickle-me-elmo-re0> request system snapshot partition
Clearing current label...
Partitioning compact-flash media (ad0) ...
Partitions on snapshot:

  Partition  Mountpoint  Size    Snapshot argument
      a      /           671MB   root-size
      e      /config     400MB   config-size
      f      /var        2GB     var-size
Running newfs (671MB) on compact-flash media  / partition (ad0s1a)...
Running newfs (400MB) on compact-flash media  /config partition (ad0s1e)...
Running newfs (2GB) on compact-flash media  /var partition (ad0s1f)...
Copying '/dev/ad1s1a' to '/dev/ad0s1a' .. (this may take a few minutes)
Copying '/dev/ad1s1e' to '/dev/ad0s1e' .. (this may take a few minutes)
The following filesystems were archived: / /config

{backup}
root@tickle-me-elmo-re0> exit   

We verify that the compact-flash is in the boot list before rebooting.

root@tickle-me-elmo-re0% sysctl machdep.bootdevs
machdep.bootdevs: usb,compact-flash,disk1,disk2,lan

root@tickle-me-elmo-re0% cli req sys reboot

*** FINAL System shutdown message from root@tickle-me-elmo-re0 ***            

System going down IMMEDIATELY         

When your router boots next time, you should be able to verify that the root “/” partition is /dev/ad0xxx (for RE-S-1800). (marked below with “<<<<<<”)

fluong@tickle-me-elmo-re0> show system storage | no-more 
Filesystem              Size       Used      Avail  Capacity   Mounted on
/dev/ad0s1a             3.5G       272M       2.9G        8%  / <<<<<<
devfs                   1.0K       1.0K         0B      100%  /dev
/dev/md0                 40M        40M         0B      100%  /packages/mnt/jbase
/dev/md1                 19M        19M         0B      100%  /packages/mnt/jkernel64-11.4R3.7
/dev/md2                 60M        60M         0B      100%  /packages/mnt/jpfe-X960-11.4R3.7
/dev/md3                5.0M       5.0M         0B      100%  /packages/mnt/jdocs-11.4R3.7
/dev/md4                 78M        78M         0B      100%  /packages/mnt/jroute-11.4R3.7
/dev/md5                 28M        28M         0B      100%  /packages/mnt/jcrypto64-11.4R3.7
/dev/md6                 45M        45M         0B      100%  /packages/mnt/jpfe-common-11.4R3.7
/dev/md7                382M       382M         0B      100%  /packages/mnt/jruntime-11.4R3.7
/dev/md8                7.9G        18K       7.2G        0%  /tmp
/dev/md9                7.9G       744K       7.2G        0%  /mfs
/dev/ad0s1e             393M        44K       362M        0%  /config
procfs                  4.0K       4.0K         0B      100%  /proc
/dev/ad1s1f              18G       1.7G        15G       10%  /var