|
Unsuitable GOS Expanded to 256KiB
Samuel A. Falvo II
kc5tja -at- arrl.net 2010 Feb 11 23:02 PDT
Unsuitable's General Object Store (GOS), used to hold all the textual content of this blog, is running out of room to hold future content. For development purposes, it was specified only for 64KiB of space, of which only 8.8KiB remained as this article was written. To handle this problem, I expanded the GOS to 256KiB. This should give me sufficient room for an estimated 68 more articles before I have to resize the GOS again. A cheat-sheet for how I administered Unsuitable follows. 1 Relocating the GOSPrior to the GOS re-allocation, the mappings.fs file contained the following: : >core dup 10 rshift block swap 1023 and + ; : blocks 1024 * ; : @f >core @ ; : !f >core ! update ; ( block 1 = meta-block ) : g-fencepost 1 block ; : a-nextId 1 block [ 1 cells ] literal + ; ( blocks 2 .. 65 = general object store ) 2 blocks constant gorg gorg 64 blocks + constant gend ( blocks 66 .. 67 = handle table for general object store ) 1 blocks constant /hfields 66 blocks constant addrs addrs /hfields + constant lens ( blocks 68 .. 77 = article table columns ) 2 blocks constant /afields 68 blocks constant articleIds articleIds /afields + constant titles titles /afields + constant leads leads /afields + constant bodies bodies /afields + constant timestamps Note that only 64 blocks served as GOS' backing store. To grow the GOS, I needed a temporary allocation, so that data migration could occur, thus giving rise to the new copy of mappings.fs: : >core dup 10 rshift block swap 1023 and + ; : blocks 1024 * ; : @f >core @ ; : !f >core ! update ; ( block 1 = meta-block ) : g-fencepost 1 block ; : a-nextId 1 block [ 1 cells ] literal + ; ( blocks 2 .. 65 = 64K unused space -- formerly general object store ) ( blocks 66 .. 67 = handle table for general object store ) 1 blocks constant /hfields 66 blocks constant addrs addrs /hfields + constant lens ( blocks 68 .. 77 = article table columns ) 2 blocks constant /afields 68 blocks constant articleIds articleIds /afields + constant titles titles /afields + constant leads leads /afields + constant bodies bodies /afields + constant timestamps ( blocks 256 .. 511 = General Object Store [256K version] ) 256 blocks constant gorg gorg 256 blocks + constant gend 2 Data MigrationI created the following code inside the util.fs file — it works by copying 64 blocks from one location (specified by f) to another (specified by t). The variables represent from and to, respectively. I use bf as a temporary holding buffer. While not efficient, it proves most effective. variable f variable t create bf 1024 allot : b f @ block bf 1024 move bf t @ block 1024 move update 1 f +! 1 t +! ; : 8b b b b b b b b b ; : movgos 2 f ! 256 t ! 8b 8b 8b 8b 8b 8b 8b 8b ; movgos After the above code executed, I removed the definition of movgos, since it wasn't needed anymore. 3 Fixing Up HandlesThe article table (defined by the articleIds, titles, leads, bodies, and timestamps columns in mappings.fs, above) references strings by way of handles. I knew, from the get-go, I'd eventually have to relocate and resize the GOS, so choosing to express strings through opaque handles proved an excellent choice, for then all I'd need to do is update the addrs column in the GOS handle table. The following code, executed interactively, proved sufficient to do the job: : ra addrs begin dup @f -1 xor while dup @f 256 blocks + over !f cell+ repeat drop ; ra After the invokation of ra (relocate addresses) completed, the Unsuitable database regained consistency, and is now safe to accept new article submissions, of which this is the first one since relocation and expansion. 4 ConclusionThis article contains a cheat-sheet for instituting capacity-planning policy changes with Unsuitable. As you can see, it's quite easy to administer Unsuitable. No, there exists no fancy GUI. Yes, you need to think logically about what you want, your posting rates, and size your table accordingly. DBAs, Hadoop administrators, and so forth do this sort of thing all the time. I invested a grand total of 10 minutes into this effort, most of which were spent refamiliarizing myself with Unsuitable's otherwise undocumented code (though, hopefully, with time, I aim to change that). |