----------------------------------------------------------------------
Synology Diskstation DS414 Unbricking
----------------------------------------------------------------------
This was many hours work so i thought it was a good idea to document properly rather than leave it online as 100 or so different forum posts (in multiple different languages) that i used to solve this tricky problem.
A special thanks to Bohdi from doozan.com and Greg Langford from greglangford.co.uk for their assistance.
----------------------------------------------------------------------
The problem
----------------------------------------------------------------------
After selling my DS on facebook marketplace (the first mistake) the guy who bought it managed to brick it. At first it seemed like the power had been pulled during an upgrade but now after recovery i have all the logs and can see he had attempted to use disks from an old DS2XX series which seemed to corrupt the boot image.
The device was delivered back at my door with a blue flashing light of death and so the project began to unbrick my DS.
The reason i say facebook marketplace was my first mistake was i couldn't find any support on what to do when a buyer returns something that they obviously broke themselves. Ebay on the other hand have processes in place to help with disputes like this. I'm not knocking facebook marketplace in general as it has been a good way to sell stuff quickly due to the amount of people it reaches in the local area but i will take this as a warning when i next come to sell something.
----------------------------------------------------------------------
The research
----------------------------------------------------------------------
So, where do you start with blue blinking light of death. Many unhelpful suggestions on the Synology forums, even from the synology support chat saying the box was “unrecoverable” but a couple of useful posts did start me on the right track.
The first one i found wasn't the most helpful but gave me hope it was fixable.
https://community.synology.com/enu/forum/17/post/69287
Now this was a scary read as soldering anything has not always worked out positively for me in the past. The important part of this post that should be carefully considered is the sentence “This guide is for all who are sure that u-boot has been corrupted and that there is no other way to recover.”
So how do i confirm if uboot is corrupted? I need to console!
Now Synology have not made this a simple plug and play process. The console pins are hidden underneath which first lead me to a youtube video on dismantling the box to find the pins. As it turns out all i needed to do was tape the correct 3 UART pins together and slide them through the correct gap in the underside of the case.
https://pasteboard.co/J46QuyY.png
I prefer the second video as until you realise which direction to push the inside bar it can be a pain to open.
**NOTE** if you are watching these thinking the DS414 can be memory upgraded then look at the photos below. The DS414 has the memory fixed to the motherboard unlike the DS415+
https://www.youtube.com/watch?v=UhsSmOLQA_g&t=135s
https://www.youtube.com/watch?v=eHCf51AytLM&t=51s
https://pasteboard.co/J46QYo5.png
https://pasteboard.co/J46RiiY.png
Once consoled i was off to a good start although I had a major pain getting tftp to work in the stock Marvel uboot.
Using a dutch post
https://gathering.tweakers.net/forum/list_message/61991482#61991482
I managed to get the DS booted but typing “save” set me back a few days as it didnt save the image correctly and meant even uboot was not loading back to its prompt. So a new search began, how to recover u-boot, hopefully without soldering or using a SOIC clip like the tweakers post suggested.
**NOTE** a SOIC clip is a clip that can attach to a rom chip for reading or writing without soldering. So for example if you couldnt console you could go direct to the chip. I think the problem for me would have been finding which chip to clip so im glad i didnt need to go down this route.
https://pasteboard.co/J46RHwE.png
Now its funny the order i found these in but the next post i came across when searching how to recover u-boot was back on the synology forum. A post by Greg Langford, most of which by this point i had done but i reached out to Greg via his website as the forum post was too old to post.
https://community.synology.com/enu/forum/17/post/59925
Clicking a link in Greg’s post led me to the doozan forum which was the most help of all.
https://forum.doozan.com/read.php?3,99414,page=2
Here I managed (with a lot of help) to load a custom uboot image using a tool called kwboot and load a custom built debian OS to reflash uboot and the zImage partitions.
I'm kinda glad it worked out that way as i learned a lot about how the Synology boots and different ways that devices running uboot can be hacked.
Using this custom debian build i restored the /dev/mtd1 partition and u-boot was back to a prompt. With a few final pointers from Greg on restoring the firmware once booted i finally had a working Synology again!
----------------------------------------------------------------------
The solution
----------------------------------------------------------------------
Step 1. Console the Synology
-- Label the disks so they go back in the correct order and remove them.
-- Remove network cable and any external USB.
-- Now console and find out why it’s not booting.
-- Does it stop before loading the image or after?
-- Do you get a Marvel prompt?
Console using a UART Cable
To console the Synology dont do what i did and open the case, just turn the case on its side and find the back 3 pins closest to the board.
https://pasteboard.co/J46Sqtv.png
The pins you’re looking for are 2,4,6 but you wont see this without the case open so tape the UART together going BLACK / WHITE / GREEN (red is not needed)
Green goes closest to the power button.
Using your favourite terminal program, set the port to the correct COM port (find the COM port for the usb serial in Device Manager) and set the baud rate to 115200. Leave everything else default. Now a tip is to turn on terminal logging so you record ALL your outputs each time you load the console. I use SecureCrt and set my scroll back buffer to 128000 and the log file to append
https://pasteboard.co/J46SBuO.png
Turn on the power while this is connected and see if you get to the Marvel prompt.
If you do then that is a good sign!
Step 2. Prepare the USB
Now you can use TFTP but to save you the issue i had i highly recommend using a usb stick.
Find a usb stick, an old one 8Gb or under is perfect. Format in windows using a tool called rufus to make sure you get the partition exact.
Format it to be non bootable, MBR (important), labelled rootfs, fat32 and quick format.
**NOTE** I used ext2 as i was preparing a custom image from a raspberry pi running raspbian and the custom uboot i was using prefered ext2.
https://pasteboard.co/J46SSda.png
Download the latest DS for your synology in pat file format and use 7zip to unpack.
https://www.synology.com/en-uk/support/download/DS414#firmware
https://pasteboard.co/J46T7tv.png
The folder you extract should look like this:
https://pasteboard.co/J46ThZFz.png
The uboot file is important if you cannot boot into a Marvel prompt (i’ll cover that later) but if you are at the prompt you just need zImage and rd.bin.
Copy these to the usb and eject.
Step 3. Boot from usb (do not save)
I made the mistake here of changing envs trying to get it to boot. The best thing here is to load the image from the usb into RAM and boot.
Here are the commands you need (Plug the drives back in first):
*** Warning - bad CRC, using default environment
Wrong Image Format for bootm command
ERROR: can't get kernel image!
So i loaded the kernel back on by downloading the latest 414 firmware from Synology, unpacking the pat file and using a USB formatted with rufus to fat16 to copy zImage and rd.bin
Marvell>> usb start
(Re)start USB...
USB: Active port: 0
Register 10011 NbrPorts 1
USB EHCI 1.00
scanning bus for devices... 2 USB Device(s) found
scanning bus for storage devices... 1 Storage Device(s) found
Marvell>> fatload usb 0:1 2000000 zImage
reading zImage
2123952 bytes read
Marvell>> fatload usb 0:1 8000000 rd.bin
reading rd.bin
3698039 bytes read
Marvell>> bootm 0x2000000 0x8000000
Step 4. Upgrade the firmware.
Hopefully this will boot back into the syno software and you can login and perform an upgrade using the pat file you downloaded.
https://pasteboard.co/J46TG83.png
View the console output here from the upgrade. It was very satisfying to watch it boot again by itself for the first time in weeks!!!
https://pastebin.com/wYBkJt8A
----------------------------------------------------------------------
What if u-boot doesn't give a Marvel prompt
----------------------------------------------------------------------
Now this was the fun part! I managed to type save as i said above which set me back a few steps as i had to find someone to help me boot a custom uboot and restore the image from debian. The fun part was seeing how this could easily be used to hack a DS into any OS from debian or even Freenas (hopefully a future guide coming soon)
Step 1. Download kwboot in Linux
Now I did all this using my raspberry pi model b. It's one of the original pis that normally sits on shelf as isn't powerful enough to run much these days but is perfect for a project like this!
The reasons i found it perfect were easy access to the usb ports as my other linux machines are all VMs plus i could start with a clean raspbian build.
I recommend the stretch version of raspian as buster seems to have a buggy version of kwboot in the main repo.
http://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2019-04-09/2019-04-08-raspbian-stretch-lite.zip
Download the zip and use rufus to load the image to the SD card ready to boot the rpi.
Boot the rpi with the new SD card and SSH from SecureCrt. Make sure its connected to the network and do an apt-get update
https://pasteboard.co/J46UO3W.png
sudo apt-get update
sudo apt-get upgrade
sudo apt install u-boot-tools minicom -y
Plug in the UART console and check which tty it has loaded as and start minicom
cat /var/log/messages | grep ttyUSB
sudo minicom --device /dev/ttyUSB0
Plug in the console to the synology and power it on. At this point my boot was hanging at
MMC: MRVL_MMC: 0
SF: Detected M25P64 with page size 64 KiB, total 8 MiB
PEX 0.0(0): Root Complex Interface, Detected Link X4, GEN 1.1
PEX 1.0(1): Root Complex Interface, Detected Link X1, GEN 2.0
PEX 1.1(2): Detected No Link.
PEX 1.2(3): Detected No Link.
FPU initialized to Run Fast Mode.
USB 0: Host Mode
USB 1: Host Mode
USB 2: Device Mode
Modules Detected:
---HANGS---
Step 2. Download the custom kwb and run kwboot
Download the kwb file for the DS414 ready to use with kwboot and winscp it across to the rpi. This is a custom u-boot that can run from UART.
kwb
Rename the file to correct the typo on the ds number and run kboot
mv u-boot-spl-2019.10-tld-1.ds114.kwb u-boot-spl-2019.10-tld-1.ds414.kwb
kwboot -t -B 115200 /dev/ttyUSB0 -b u-boot-spl-2019.10-tld-1.ds414.kwb -a
Sending boot message. Please reboot the target...-
Now power on the Synology and it should boot into a custom u-boot with a => prompt
https://pastebin.com/qZnHr1Sv
Step 3. Prepare the custom debian os usb stick
https://forum.doozan.com/read.php?2,32146
Format the usb as ext2 this time as this custom u-boot doesnt have ext3 or 4 loaded and seemed a little unstable using fat. Make sure its MBR not GPT and the label is rootfs
https://pasteboard.co/J46Wbyk.png
Download kernel and winscp it over to the rpi or wget direct into the home dir
https://bitly.com/37ygLsI
The run the following (assuming your usb is /dev/sda)
sudo su -
mount -t ext2 /dev/sda1 /media/usb/
cd /media/usb
tar -xvjf /home/pi/Debian-5.2.9-mvebu-tld-1-rootfs-bodhi.tar.bz2
cd boot
cp -a zImage-5.2.9-mvebu-tld-1 zImage.fdt
cat dts/armada-xp-synology-ds414.dtb >> zImage.fdt
mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n Linux-5.2.9-mvebu-tld-1 -d zImage.fdt uImage
sed -i 's/ext3/ext2/' /media/usb/etc/fstab
cp -v /home/pi/uboot_DS414r1.bin /media/usb/root/
cp -v /home/pi/zImage /media/usb/root/
cd;sync;umount /media/usb
**NOTE** The lines below are from the pat file from the synology website. I used these to reflash mtd0 and mtd1 which got my Synology booting back to the Marvel prompt.
cp -v /home/pi/uboot_DS414r1.bin /media/usb/root/
cp -v /home/pi/zImage /media/usb/root/
Step 4. Boot into Debian from usb
Move the usb stick into the FRONT usb next to the power button. Use the kwboot command again to boot to the => prompt and run the following:
kwboot -t -B 115200 /dev/ttyUSB0 -b u-boot-spl-2019.10-tld-1.ds414.kwb -a
setenv bootargs 'console=ttyS0,115200 root=/dev/sda1 rootdelay=3 mtdparts=spi0.0:0x000d0000(u-boot),0x002d0000(zimage),0x00430000(rd),0x00010000(vendor),0x00010000(u-boot-envs),0x00010000(fis) earlyprintk=serial'
ext2load usb 0:1 2000000 boot/uImage
bootm 0x2000000
If this boots correctly the output should be similar to this.
https://pastebin.com/CHNGv0Sa
Step 5. Login to Debian and backup the mtd partitions
Login with root / root
Check the mtd parts have the u-boot labels NOT the RedBoot labels.
root@debian:~# cat /proc/mtd
dev: size erasesize name
mtd0: 000d0000 00001000 "u-boot"
mtd1: 002d0000 00001000 "zimage"
mtd2: 00430000 00001000 "rd"
mtd3: 00010000 00001000 "vendor"
mtd4: 00010000 00001000 "u-boot-envs"
mtd5: 00010000 00001000 "fis"
Backup each of these parts to usb. The counts are hex to decimal conversions of the above.
root@debian:~# dd if=/dev/mtd3 of=mtd3 bs=1 count=65536
root@debian:~# dd if=/dev/mtd4 of=mtd4 bs=1 count=65536
root@debian:~# dd if=/dev/mtd5 of=mtd5 bs=1 count=65536
root@debian:~# dd if=/dev/mtd0 of=mtd0 bs=1 count=851968
root@debian:~# dd if=/dev/mtd1 of=mtd1 bs=1 count=2949120
root@debian:~# dd if=/dev/mtd2 of=mtd2 bs=1 count=4390912
Step 6. Restore the stock u-boot and zImage
Now i got an error here verifying the data but this got me back to a working Marvel prompt where i was able to boot zImage and rd.bin from usb like the first part of this guide.
If this works for you without error then it’s possible that you won’t need any further steps and a reboot will boot you back into the stock syno OS.
root@debian:~# flash_unlock /dev/mtd0
root@debian:~# flashcp -v uboot_DS414r1.bin /dev/mtd0
Erasing blocks: 203/203 (100%)
Writing data: 808k/808k (100%)
Verifying data: 808k/808k (100%)
root@debian:~#
root@debian:~# flash_unlock /dev/mtd1
root@debian:~# flashcp -v zImage /dev/mtd1
Erasing blocks: 519/519 (100%)
Writing data: 2074k/2074k (100%)
Verifying data: 2050k/2074k (98%)File does not seem to match flash data. First mismatch at 0x001fe000-0x00200800
Good luck.