Systems & Stories

Next couple of blogs are going to be centered around Embedded Linux boards because that’s something that has always fascinated me. One of the best boards in the market with huge adoption, excellent documentation, open DTBs, detailed datasheets, and a very active open-source ecosystem is the BeagleBone Black — so naturally, I bought one.

It has always been fun messing around with boards, kernels, bootloaders and random hardware bring-up experiments. But what truly goes on from the instant power is applied to a fully operational Linux system is far more complex than most people realize. There’s an entire chain of events involving ROM code, SRAM, DDR initialization, PLL stabilization, memory calibration, bootloaders, kernel loading, and finally userspace startup.

The entire flow on the BeagleBone Black looks roughly like this:

At a very high level, power is first applied to the board, after which the AM335x Boot ROM begins executing. The ROM loads the SPL/MLO into SRAM, which then initializes clocks, DDR, PMICs and other low-level hardware. Once external DDR becomes usable, full U-Boot is loaded into memory, followed by the Linux kernel, device tree and finally the root filesystem before Linux userspace comes alive. The Boot ROM itself is burned directly into the AM335x SoC by Texas Instruments and is immutable by design. Since TI manufactures only the SoC and not the final board, the ROM has no knowledge about the exact DDR layout, PMIC configuration, GPIO routing, peripherals or memory timings used by a specific board like the BeagleBone Black. It intentionally performs only minimal initialization required to locate and load a secondary bootloader into SRAM.

That responsibility falls onto the board-specific SPL/MLO and U-Boot code provided by projects like BeagleBone. In other words, the actual usable boot sequence is really ROM → MLO/SPL → U-Boot → Linux Kernel → RootFS

My current development setup looks something like this:

The SD card only contains the earliest bootloader stages (MLO and u-boot.img). The actual Linux kernel (uImage) and Device Tree are served over TFTP from the host machine, while the root filesystem is mounted over NFS. UART logs are captured through a USB-UART adapter connected to the host machine so that every stage of the boot process can be monitored live.

This setup is incredibly convenient because kernel iterations become extremely fast. Device Trees can be modified instantly without reflashing SD cards, rootfs changes appear immediately over NFS, and complete boot logs are always visible over UART. Once the Boot ROM loads the SPL into SRAM, the real board bring-up begins. At this stage, the system is still operating in an extremely primitive state. DDR is not initialized yet, most peripherals are unavailable, the normal serial subsystem does not exist yet, and only a tiny amount of SRAM is available for execution. That means debugging this stage is not as simple as adding a printf() somewhere in the code. To enable very early UART debugging support inside SPL, I enabled the following U-Boot configuration options:

CONFIG_DEBUG_UART=y
CONFIG_DEBUG_UART_OMAP=y
CONFIG_DEBUG_UART_BASE=0x44e09000
CONFIG_DEBUG_UART_CLOCK=48000000
CONFIG_DEBUG_UART_SHIFT=2

These enable a minimal UART backend that works before the full serial driver stack is initialized. Without this, early SPL debugging would effectively be blind. After enabling these and sprinkling debug logs throughout the SPL code, the actual early boot flow became visible:

This was probably the most fascinating part of the entire experiment because you can literally watch the SoC slowly “wake up” in real time. First the watchdog gets disabled so the board does not reset during bring-up. Then UART pinmuxing appears, clocks become active, PLLs stabilize, PMIC voltages get configured, DDR PHY calibration begins and eventually external RAM becomes usable. One thing that immediately caught my attention in this sequence was the DDR initialization path. A lot of questions started appearing naturally: – How is time even measured this early? – What exactly is a DPLL? – How are DDR clocks generated? – Where do DDR timing values come from? – Why are timings so board specific?

The BeagleBone Black starts with a very simple 24 MHz crystal oscillator physically present on the board:

But DDR memory obviously cannot operate at just 24 MHz. To generate the required high-speed clock, the AM335x uses a DPLL (Digital Phase Locked Loop), which multiplies the reference clock frequency. In the case of the BeagleBone Black DDR subsystem: 24 MHz × 16.67 ≈ 400 MHz So the DDR subsystem ultimately operates at roughly 400 MHz. This explains the DDR PLL and clock generation part. But the far more interesting topic is what happens after the clock becomes stable because having a 400 MHz clock alone is not enough. The controller still needs to know: – how long rows need to stay active – when reads are allowed – how frequently refreshes happen – how long writes take – how quickly rows may be switched – when precharge operations complete

That entire behavior is controlled through the EMIF timing registers. For the BeagleBone Black, the DDR timing configuration structure looks something like this:

These are not arbitrary hexadecimal constants copied from somewhere on the internet. Each register contains multiple tightly packed DDR timing parameters encoded into specific bitfields.

For example: sdram_tim1 = 0x0aaad4db contains fields for: – tRCD – tRP – tRAS – tWR – tRRD – and several other DDR timing constraints

One particularly interesting register is the refresh controller. DRAM cells store information as tiny electrical charges inside capacitors. Those charges naturally leak away over time, which means memory rows must periodically be refreshed or the stored data disappears. On the AM335x EMIF controller, refresh timing is configured through ref_ctrl.

The stock BeagleBone Black configuration uses `ref_ctrl = 0x00000c30`.
The programmed refresh counter value becomes:
0x0c30 = 3120 cycles
Since DDR operates at 400 MHz:
1 cycle = 2.5 ns
Therefore:
3120 × 2.5 ns = 7.8 us

meaning the EMIF performs a refresh operation roughly every 7.8 microseconds. Naturally, the next thought was what happens if I intentionally break these timings? Surprisingly, increasing the refresh interval significantly did not immediately crash the board. I pushed the refresh timing as high as 161.4 us and the system still continued operating. I stress-tested memory using both mtest and memtester, but the board remained surprisingly stable.

So I moved on to a much more critical timing parameter: tRCD. tRCD stands for RAS-to-CAS Delay and defines how long the controller must wait after activating a DRAM row before issuing a READ or WRITE command. Inside SDRAM_TIM_1, the tRCD field is encoded in bits [24:21]. From 0x0aaad4db the encoded value becomes 5. The AM335x EMIF timing fields follow this encoding rule: actual value = encoded value + 1.

Therefore:
tRCD = 5 + 1 = 6 cycles
At 400 MHz DDR:
1 / 400,000,000 = 2.5 ns
which means:
tRCD = 6 × 2.5 ns = 15 ns

So the EMIF waits roughly 15 ns after opening a DRAM row before attempting to access data from it. At first, I reduced this from 6 cycles → 5 cycles which changes the delay from 15 ns → 12.5 ns and surprisingly the board still booted successfully. But once I reduced it further 6 cycles → 3 cycles which becomes 15 ns → 7.5 ns the board finally became unstable and crashed shortly after full U-Boot started. That behavior makes perfect sense because reducing tRCD too aggressively causes the EMIF controller to issue READ or WRITE commands before the DRAM row has fully stabilized internally. At that point, memory accesses become timing-unsafe and corruption starts appearing. One interesting observation was that SPL itself still continued functioning even with broken DDR timings. The reason is fairly simple: SPL barely uses DDR compared to full U-Boot. SPL primarily executes from SRAM, uses very small buffers and performs relatively few memory accesses. Full U-Boot, however, relocates itself entirely into DDR, enables networking, loads kernels and starts stressing memory much more heavily. That increased memory pressure is what finally exposes unstable DDR timings.

Stay Tuned for More :)

By – Shaurya TW: AI Assistance has been used for image gen and some paraphrasing of texts.