Can a retro-computer be a useful way to learn computer-architecture fundamentals?

The main benefit for learning computer system architecture with a retrocomputer, compared to a modern computer, is accessibility of the hardware. Retrocomputers are much simpler than modern systems in terms of both the hardware and the system software. By design and necessity, programming on a retrocomputer means often programming the bare metal, and that means understanding how the machine works in exquisite detail. At the same time, you get the aid of a big software library (often including a built-in BASIC interpreter) that shows off what's possible with the hardware, plus a fully self-contained environment to program in, plus a simple hardware architecture that can easily support hardware hacking too.

The commenter above makes a very valid point about Arduino, which also is designed for this sort of bare metal software hacking and building your own hardware add-ons. I think the thing you would miss out on is the fascinating history of the retrocomputers, and the fact that they are normally far more sophisticated than just the CPU+GPIO that you get with Arduino. Many retrocomputers have unique and powerful coprocessors for sound and graphics. At minimum, they all support basic peripherals like keyboard, mouse/joystick, and video display. You'd have to attach some primitive proxy for these I/O device to Arduino to make it a full, self-contained computer system that you can program or use stand-alone. Then again, for programming efficiency, having something tethering you to a modern device can be very helpful. And that is possible to do and equally easy with either Arduino or a retrocomputer.

The original Apple ][ is an expensive retrocomputer because it is rather rare and collectible. But you can acquire an Apple //e, //c, or IIgs for a much more reasonable price. Of all of these, the IIgs is the most sophisticated and probably offers the most long-term hacking potential. But you could get very far "down the rabbit hole" with any of them. And, I also think you'd be happy with other popular 1980s 8-bit machine like Commodore, Atari, IBM 5150, or Tandy (showing my North American persuasion). Regardless, be sure to get a machine that comes with or can be adapted to a video display that you have available, and also investigate the supported hobbyists projects that will help you move files and disk images between the modern world and your retrocomputer of choice.