Good morning/afternoon, everyone. My name is Samuel Falvo, and I’m here today to give you an update on a project of mine which has been in the works in one way or another since 2004.
Back in 2004, I was a regular member of the 6502 forums. It was almost a right of passage for people there to build their own computers. Of course, I was no exception.
The schematic you see here was built on a pair of Radio Shack breadboards. You’ll see I’m using a Western Design Center 65816 processor clocked at 4MHz. It is talking to a single 32 kilobyte RAM chip (mapped where ROM normally goes), and a single 65C22 Versatile Interface Adapter or VIA chip (mapped where RAM normally sits). Although the schematic shows a 74LS00 for address decoding, I believe I actually used a 74HCT00 instead, since everything else in the circuit is CMOS. I used a separate circuit, called the IPL circuit, to upload the software into the RAM chip for execution via my PC using DMA.
This circuit worked wonderfully. Still, it’s ultimately not what I wanted. Its only connection to the outside world was this VIA chip. With what I know today, I could’ve made this work for me; but, back then, I lacked the understanding and tooling to make use of what I had. Plus, I wanted something closer to a Jupiter Ace, something with more finesse to it.
What I wanted was something that had, say, 512KB of RAM mapped in low memory, where RAM should actually be. This is a nice figure, since you can now conveniently acquire such memory capacities at reasonable cost for a single chip. I wanted Forth to be the system programming language and operating system for the device. I didn’t want to be tethered to my PC; that rather defeated the purpose. And, I wanted I/O to be message-passing yet hacker friendly, allowing me to be maximally efficient at building actual hardware and software. PS/2 keyboards and mice will some day disappear; my thinking was that I should be able to replace PS/2 hardware with USB hardware without affecting already built core components, both hardware and software.
Oh, and of course, I wanted good graphics. The 128-tile limitation that the Jupiter Ace supported seemed, while understandable for its time, ungainly and limiting in the 21st century.
Over the years that have elapsed since, various bits and bobs of the Kestrel-2, Kestrel-2DX, and Kestrel-3 projects evolved to focus on one or more of my core desires for the ForthBox. The MGIA, monochrome graphics interface adapter, core of the Kestrel-2 provided pure bitmapped graphics with only a 16KB investment in RAM. The GPIA, general purpose interface adapter, provided greater I/O capability. Yet, I never managed to come closer to my most desired goal.
Well — 19 years have come and gone; time to finish the damned project.
Regrettably, no actual working hardware exists anymore. The old breadboards have long since been disassembled and recycled for other projects. Even if there was, the old circuit is clearly too limiting. It would’ve needed a total redesign no matter what.
We’d need to redesign the circuit to capture the bank address byte, exposing the entire 16MB address space the CPU is capable of. Next, we’d need to relocate I/O resources out of zero-page, so that RAM can go there instead. We’d need to install real ROMs to bootstrap the device, instead of relying on a janky IPL circuit.
As of 2022, this could have been accomplished, were it not for the silicon shortage caused by COVID-19 and greedy automotive manufacturers. However, even with the two years I’d’ve had remaining, the finished product still would not have had any system software to run on it.
So, instead, I’ve been focusing my effort on a software emulator which currently runs under Linux. This brings several benefits:
The emulator currently has these features:
As the project evolves, these features may change; but, for now, this is what’s available.
The emulator runs under Linux. It ought not be too hard to port the emulator to other platforms; I just lack the facility to do so at the moment.
Now is a good time to demo the ForthBox emulator.
(live demo)
Firmware consists of a simple port of the KIM-1 TIM interactive monitor to the 65816, and actually exists independently of the System Forth firmware image. It’s actually set up as a kind of crude hypervisor of sorts, using the BRK instruction as a system call mechanism.
We instruct the computer to jump into System Forth image by setting the PC to its origin, and typing G to go.
As you can see, the overall user experience is pretty standard for a Forth environment. I’m almost ready to let Forth take over from TIM on power-on reset, but there are still some bugs to work out.
Virtual storage and virtual terminal are IDEAL, and do not represent realistic hardware yet. Blocks are completely supported as far as I can tell; however I still need to write an editor. Wordlist manipulation frequently crashes the environment; I’m not sure why yet, so the editor will need to wait. Also, System Forth cannot yet forget words; MARKER not defined yet. Not quite sure how to make it all play nice together. So, there’s still work to go yet.
Finish in 1.5 years? … no. Not going to happen.
Priorities have shifted significantly since I last spoke at SVFIG. This is my version of Maslow’s hierarchy of needs; only, in my case, it’s an allocation of spoons. As you can see, family concerns have become an increasingly demanding drain on my spoons. My in-laws each have had their own impactful medical issues, and we’ve recently had an aunt pass away. That, in turn, has made me a home owner now. So I’m having to deal with the financial and investment responsibility for that property. We have pets that are getting up in age, and are showing signs of age-related illnesses. And, of course, my employer is currently in something of a crunch-mode at the moment, which further requires my undivided attention during the day. So after everything is said and done, I usually don’t have all that many spoons left to work on personal projects. Except for the occasional “I’m going to spend a day or two hacking on a GUI because … well, just because” type project. I never get very far with such projects; but they always somehow seem cathartic regardless.
Any time spent on this project must, therefore, be maximally efficient. Which actually brings me, finally, to the original intended topic of this talk.
The Shoehorn Retargetable Compiler. The whole purpose of this compiler is to rapidly bring up a new hardware platform from bare metal. How rapidly? How about a week? That’s roughly how long it took me to go from nothing to printing debug messages to the console.
System Forth is written almost entirely in Shoehorn. A small selection of which is visible here. You’ll notice the colon definitions have strange syntax to them. That’s because I frequently need not one, but two, names for a definition. The first name conforms to the syntax used by an assembler; the second name can be any valid Forth name. This is because, unlike most other metacompilers, Shoehorn does not compile into a raw binary image. Rather, it outputs a textual assembly language, which can be processed by other tools later on.
Here’s what the 65816 back-end currently looks like. I won’t get into the details here, but all constructs in Forth ultimately decompose into one or more calls into these 20 or so primitives, which we see on the far right. There’s like three more which isn’t shown here.
Just behind or above the primitive entry points, we see a bunch of utilities to generate 65816 assembly language output.
And behind/above that, some support code for maintaining an 8-way hashed dictionary structure.
Please note that these primitives are not the same thing as the 30 or so primitives that Chuck Moore and others identified for a minimal Forth. Although there is a vague correspondance on some things, these primitives are for compilation of runtime semantics of Forth, not interpretation of core primitives.
In fact, as far as Shoehorn is concerned, there is literally no difference between a core primitive and a human-programmed colon definition. Note these XREF and XNAME declarations here. These tell Shoehorn, “Don’t throw up an error when these names are referenced; trust me, I have defined them myself elsewhere.” Kind of like “extern” in C or C++.
And here, we see where I have defined the runtime behavior of some of these primitives.
So, as people may know by now, I am a fan of the RISC-V architecture, especially the 64-bit specification. How easy would it be for me to port System Forth to the RISC-V architecture?
First, you’d need to be aware of all the code in your program that is sensitive to things like endianness and stack width, and replace them with something more portable. Yes, this includes when saving things into block storage. I learned this lesson the hard way, back when I upgraded my GForth from a 32-bit to a 64-bit version, and it destroyed my blog engine’s message base by accident.
By the way, because I know Youtube commenters will bring this up, this affects C and C++ programmers just as much as it would affect Forth as well, so this should be nothing new to anyone. Rob Pike even mentions this in multiple papers he wrote while working on the Plan 9 from Bell Labs operating system.
But, once that has been done, you should, in theory, just have to write a new Shoehorn back-end, and you’d be good to go.
Here’s a 64-bit RISC-V back-end for an older version of Shoehorn. It used to be complete some years ago, around the time of Shoehorn 1.0; but, it isn’t complete anymore. I haven’t had a need to play with the RISC-V back-end since I first wrote Shoehorn, so I never bothered to update it. I will eventually.
But you can clearly see how little code investment it takes to retarget Shoehorn to a radically new processor architecture. This listing shows about 10 entry points. The 65816 backend you saw earlier had only about 20 or so. The entry points are documented in the README.md file for the compiler.
But, then, that’s the point. That’s why I wrote Shoehorn. The mission was not to have the fastest possible software after final assembly and linking. The mission was to be maximally efficient given my extremely limited time resources. To produce the largest amount of progress in a project I can make with days of effort. This philosophy applies equally to both System Forth and to Shoehorn; it isn’t uncommon to have to maintain Shoehorn in response to discovering a new need during project development.
Recapitulating, it is not just relatively easy to write a new target; it is easy to write a new target. It takes a single developer hours to days to write a new back-end from scratch; not a whole team effort of months to years like other compiler back-ends.
Here’s how Shoehorn fits into the greater development work-flow. Like any old-school compiler technology, it sits directly in front of an external assembler.
This is what an invokation of Shoehorn looks like in practice. This is the script I use to rebuild System Forth entirely from scratch. Most of this is boilerplate for the benefit of the emulator, setting up virtual storage images, boot sectors, all that stuff.
Observe how the output of Shoehorn is referenced via an include directive in the bottom of the primitives source listing. This is how everything gets linked together to form the finished binary image. This isn’t the only way of doing things; with more sophisticated tooling, you can build static libraries, shared libraries, etc. Whatever is appropriate for your project at hand.
I’m extremely pleased with Shoehorn. This is my ColorForth, if you will. I plan on using it to produce all manner of things in the future.
Maybe in the future, I can finally figure out how to use more conventional metacompiler words like TARGET and HOST to control Shoehorn so that I don’t need as much customized syntax. But, even as it currently is, it is every bit as productive to me as any C compiler would be. And if my experience at work is any indication, arguably more so.
That’s all I have for today. Here are links to the relevant pieces of software, including both Shoehorn and an emulator for the 20th anniversary edition of the ForthBox.
I’ll now take questions if there are any.