I've had a number of great guests on the podcast lately. One topic that has come up a number of times is the "toy project." I've usually kept mine private - never putting them on GitHub - Somewhat concerned that people would judge me and my code. However, hypocrite that am (aren't we all?) I have advocated that others put their "Garage Sale Code" online. So here's some crappy code. ;)
The Preamble
While I've been working as an engineer for 25 years this year, I didn't graduate from school with a 4 year degree until 2003 - I just needed to get it done, for myself. I was poking around recently and found my project from OIT's CST352 "Operating Systems" class. One of the projects was to create a "Virtual CPU and OS." This is kind of a thought exercise. It's not really a parser/lexer - although there is both - and it's not a real OS. But it needs to be able to take in a made-up quasi-Assembly Language instruction set and execute them on a virtual CPU while managing virtual memory of arbitrary size. Again, a thought exercise made real to confirm that the student understands the responsibilities of a CPU.
Here's an example "application." Confused yet? Here's the original spec I was given in 2002 that includes the 36 instructions the "CPU" should understand. It has 10 general-purpose 32bit registers address as 1 through 10. Register 10 is the stack pointer. There are two bit flag registers - sign flag and zero flag.
Instructions are "opcode arg1 arg2" with constants prefixed with "$."
11 r8 ;Print r8
6 r1 $10 ;Move 10 into r1
6 r2 $6 ;Move 6 into r2
6 r3 $25 ;Move 25 into r3
23 r1 ;Acquire lock in r1 (currently 10)
11 r3 ;Print r3 (currently 25)
24 r1 ;Release r4 (currently 10)
25 r3 ;Sleep r3 (currently 25)
11 r3 ;Print r3 (currently 25)
27 ;Exit
I write my homework assignment in 2002 in the idiomatic C# of the time on .NET 1.1. That means no Generics<T> - I had to make my own strongly typed collections. That means C# has dozens of (if not a hundred) language and syntax improvements. I didn't use a Unit Testing Framework as TDD was just starting around 1999 during the XP (eXtreme Programming) days and NUnit was just getting start. It also uses "unsafe" to pin down memory in a few places. I'm sure there are WAY WAY WAY better and more sophisticated ways to do this today in idiomatic C# of 2017. Those are excuses, the real reasons are my own ignorance, ability, combined with some night-school laziness.
One of the more fun parts of this exercise was moving from physical memory (a byte array as I recall) to a full-on Memory Manager where each Process thought it could address a whole bunch of Virtual Memory while actual Physical Memory was arbitrarily sized. Then - as a joke - I would swap out memory pages as XML! ;) Yes, to be clear, it was a joke and I still love it.
You can run an "app" by passing in the total physical memory along with the text file containing the program, but you can also run an arbitrary number of programs by passing in an arbitrary number of text files! The "TinyOS" will handle each process thinking it has its own memory and will time
If you are more of a visual learner, perhaps you'd prefer this 20-slide PowerPoint on this Tiny CPU that I presented in Malaysia later that year. You dig those early 2000-era slides? I KNOW YOU DO.
Updating a .NET 1.1 app to cross-platform .NET Core 2.0
Step 1 was to download the original code from my own blog. ;) This is also Reason #4134 why you should have a blog.
I decided to use Visual Studio 2017 to upgrade it, and even worse I decided to use .NET Core 2.0 which is currently in Preview. I wanted to use .NET Core 2.0 not just because it's cross-platform but also because it promises to have a pretty large API surface area and I want this to "just work." The part about getting my old application running on Linux is going to be awesome, though.
Visual Studio then pops a scary dialog about upgrading files. NOTE that another totally valid way to do this (that I will end up doing later in this blog post) is to just make a new project and move the source files into it. Natch.
Visual Studio says it's targeting .NET 2.0 Full Framework, but I ratchet it up to 4.6 to see what happens. It builds but with a bunch of errors about Obsolete methods, the most interesting one being this one:
Warning CS0618
'ConfigurationSettings.AppSettings' is obsolete:
'This method is obsolete, it has been replaced by
System.Configuration!System.Configuration.ConfigurationManager.AppSettings'
C:UsersscottDownloadsTinyOSOLDOLDOS ProjectCPU.cs 72
That's telling me that my .NET 1/2 API will work but has been replaced in .NET 4.x, but I'm more interested in .NET Core 2.0. I could make my EXE a LIB and target .NET Standard 2.0 or I could make a .NET Core 2.0 app and perhaps get a few more APIs. I didn't do a formal analysis with the .NET Portability Analyzer but I will add that to the list of Things To Do. I may be able to make a library that works on an iPhone - a product that didn't exist when I started this assignment. That would be Just Cool(tm).
I decided to just make a new empty .NET Core 2.0 app and copy the source .cs files into it. A few interesting things.
- My app also used "unsafe" code (it pins memory down and accesses it directly).
- It has extensive inline documentation in comments that I used to use NDoc to make a CHM Help file. I'd like that doc to turn into HTML at some point.
- It also has an appsettings.json file that needs to get copied to the output folder when it compiles.
- While I could publish it to a self-contained .NET Core exe, for now I'm running it like this in my test batch files - example:
- dotnet netcoreapp2.0/TinyOSCore.dll 512 scott13.txt
Here's the resulting csproj file.
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.0</TargetFramework>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
</PropertyGroup>
<PropertyGroup>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
</PropertyGroup>
<ItemGroup>
<None Remove="appsettings.json" />
</ItemGroup>
<ItemGroup>
<Content Include="appsettings.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Configuration" Version="2.0.0-preview2-final" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="2.0.0-preview2-final" />
<PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="2.0.0-preview2-final" />
<PackageReference Include="Microsoft.Extensions.Options.ConfigurationExtensions" Version="2.0.0-preview2-final" />
</ItemGroup>
</Project>
Other than the obsolete configuration warning and a few malformed XML comments, the app compiled and ran! You can actually "watch" the nightmare process here https://github.com/shanselman/TinyOS/commits/Core2Port in the form of GitHub commits. I also moved the docs from a 2002 Word Doc to Markdown so be sure to explore the fairly extensive spec https://github.com/shanselman/TinyOS.
The only significant change was loading the config. Configuration is even more different on .NET Core 2.0 than Full Framework. It's FAR more, ahem, configurable. I could have used "Options," I could have written my own config provider if it was important to keep the file format.
This little TinyOS has a bunch of config options that come in from a .exe.config file in XML like this (truncated):
<configuration>
<appSettings>
<!--
Must be a factor of 4
This is the total Physical Memory in bytes that the CPU can address.
This should not be confused with the amount of total or addressable memory
that is passed in on the command line.
-->
<add key="PhysicalMemory" value="128" />
<!--
Must be a factor of 4
This is the ammount of memory in bytes each process is allocated
Therefore, if this is 256 and you want to load 4 processes into the OS,
you'll need to pass a number > 1024 as the total ammount of addressable memory
on the command line.
-->
<add key="ProcessMemory" value="384" />
<add key="DumpPhysicalMemory" value="true" />
<add key="DumpInstruction" value="true" />
<add key="DumpRegisters" value="true" />
<add key="DumpProgram" value="true" />
<add key="DumpContextSwitch" value="true" />
<add key="PauseOnExit" value="false" />
I have a few choices. I could make a Configuration Provider and reach .NET Core to read this format (there's an XML adapter, in fact) or make the code porting easier by moving these "name/value" pairs to a JSON file like this:
{
"PhysicalMemory": "128",
"ProcessMemory": "384",
"DumpPhysicalMemory": "true",
"DumpInstruction": "true",
"DumpRegisters": "true",
"DumpProgram": "true",
"DumpContextSwitch": "true",
"PauseOnExit": "false",
"SharedMemoryRegionSize": "16",
"NumOfSharedMemoryRegions": "4",
"MemoryPageSize": "16",
"StackSize": "16",
"DataSize": "16"
}
This was just a few minutes of search and replace to change the XML to JSON. I could have also written a little app or shell script. By changing the config (rather than writing an adapter) I could then keep the code 99% the same.
My code was doing things like this (all over...there was no DI container yet):
bytesOfPhysicalMemory = uint.Parse(ConfigurationSettings.AppSettings["PhysicalMemory"]);
And I'd like to avoid major refactoring - yet. I added this bit of .NET Core configuration at the top of the EntryPoint and saved away an IConfigurationHost:
var builder = new ConfigurationBuilder()
.AddJsonFile("appsettings.json");
Configuration = builder.Build();
I've got a Dictionary in the format of the IConfiguration host called "Configuration." So now I just do this in a dozen places and the app compiles again:
bytesOfPhysicalMemory = uint.Parse(Configuration["PhysicalMemory"]);
This brings up that feeling we all have when we look at old code - especially our own old code. I should have abstracted that away! Why didn't I use an interface? Why so many statics? What was I thinking?
We can beat ourselves up or we can feel good about ourselves and remember this. The app worked. It still works. There is value in it. I learned a lot. I'm a better programmer now. I don't know how far I'll take this old code but I had a lovely afternoon porting it to .NET Core 2.0 and I may refactor the heck out if it or I may not.
For now I did update the smoke tests to run on both Windows and Linux and I'm happy with the experiment.
Related Links
- Download PPT Slides on the Tiny OS presented at TechEd Malaysia 2002
- Andy Clarke from New Zealand took the original spec and did the homework assignment in 2012! His project - 5 years ago, and 10 years after mine - includes some interesting changes. Rather than an EXE that takes in the programs from the command line, he's written over 221 NUnit 2 tests that check each individual component he's written as well as more comprehensive integration tests (as unit test) for the programs. The "Assembly" language has been changed from opcodes to more human readable commands like "move" and "add." I think Andy's solution is much nicer than mine, but he wouldn't pass the class because the spec was pretty clear and my teacher was a stickler. ;) I LOVE that someone else did this on their own!
- PODCAST: YOU should write an interpreter with Thorsten Ball
Have YOU done a project like this, either in school or on your own?
Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test, build and debug ASP.NET, .NET Framework, .NET Core, or Unity applications. Learn more and get access to early builds!
© 2017 Scott Hanselman. All rights reserved.
