Reminiscing, removing dependencies and preparing to optimise.

Interesting to go back and read my blog posts from the beginning! So much that I'd forgotten about, and quite obviously a scary amount of hours put into retro ports thus far. Doesn't seem like I have a lot to show for it though... 😒

I didn't read them all today, but I will have to go back and read all my entries for Donkey Kong when the time comes. If the Lode Runner and Knight Lore posts are anything to go by, it will go a long way towards reminding me of what I've done and how I've done it. No doubt I get a lot more out of my own blogs than anyone else reading them...

So back to Xevious today and I finally removed the dependencies on the Neo Geo layer in the main code. All the hit-box calculations are done using the sprite shadow register values which are - for efficiency reasons - in OSD (Neo Geo) format. In order to finish the transcode ASAP, I simply left it that way, with a note to revisit it when it was time to optimise.

All the hit-box calculations use the 8 most significant bits of the X/Y register. On the original hardware, the most significant bit of the X register had to be rotated in via the carry to produce X[8:1] every time it was referenced in a hit-box calculation. On the Neo Geo, X had to be shifted down by 8 bits, while Y had to be shifted down by 7 bits. Neither solution is optimal, so I opted to create another (platform-agnostic) copy of the X/Y register values exclusively for hit-box calculations (I already had to create a 2nd copy of X a while back to handle 2x2-tile sprites anyway).

Since this is done as the last operation in the SUB CPU main loop before spinning waiting on VBLANK, timing isn't critical. This also had the added bonus of simplifying the 68K code whenever they were referenced, as I combined the previously separate X&Y values into a single array, and eliminated the need to do word operations or any pre-shifting.

It should also allow other platforms (Amiga) to start implementing sprites now.

Now I can focus solely on getting the Neo Geo port to run at 100% with no disappearing sprites. The few remaining bugs will sort themselves eventually. I haven't actually profiled any of the code yet, but I do know the critical sections will be scrolling and sprite h/w updates, as they're both done during VBLANK. Scrolling is currently horrendously inefficient, so I'm hoping for big gains there.

Foreground and background access routines are pretty horrid as well - and they disable interrupts whilst executing by necessity - but there's not a lot of that happening during the game. So I'm not sure how much can be done there.

For disappearing sprites (96 per scanline limitation) I've thought a little about how I can mitigate that; by ensuring that unused sprites are inactive (in the Neo Geo h/w) and also turning off some foreground sprites that aren't used during gameplay.

The fun is over. Now let the fun begin!

New Year check-in. Holiday is over and now it's time to recover from it! Will take a few days to get back into a routine. I also have to sort out my dead hard disk and set up my desktop computer - yet again.

No more work on Xevious since the last update but instead I did a bit of work on Donkey Kong. I updated the tools to convert the graphics and changed the rotation of all the tiles to match Xevious. I also updated the project structure in preparation for separating the core and Neo Geo code in the same way I have done for Xevious. I'm not sure how much work this will entail as I haven't looked at my Donkey Kong code for exactly 9 years now.

As it stands, Donkey Kong is running as I left it all those years ago, with the new rotation and project structure. And here it will probably remain while I revert my focus back to Xevious in the coming weeks.

Quick 'n' dirty fixes to get it running again

As I mentioned in earlier posts, time to fix a couple of minor bugs in the transcode and then onto wholesale optimisation of the Neo Geo code. And while that is happening, hopefully the Amiga port will start to take shape...

The core running on the Amiga, showing the foreground tile layer

Stay tuned!
 

A major milestone - on Xmas Eve no less!

I have finally reached a major milestone in the project; with the debugging of the high score entry screen, the transcoding from Z80 to 68K is essentially complete! 😀

High Score entry screen complete to round out the transcode!

I do know of a couple of minor bugs in the transcode, and lack of proper support for flipping of 2x2-tile sprites, but I've been focusing on getting the remainder of the code RE'd and converted to 68K.

Next task is to go back through the listing for both side-by-side, add or clarify any missing annotations, fix typos in labels etc, and identify and document any unused code. I also need to tidy up the memory variable area(s) and ensure they're initialised (to zero) in the right places. It's a bit of a dog's breakfast in the original code as there are so many different areas of shared RAM.

Then it'll be onto Neo Geo optimisation and getting it running at 100% on real hardware, with little or no disappearing (Neo Geo) sprites (due to scanline limit). Somewhat fortuitously, visible display masks and the foreground tile layer are the first to disappear, so at least the 'sprites' won't be affected. But of course ideally there will be no disappearing (Neo Geo) sprites at all!

I'm also working with another developer to get this running on the Amiga! Once the ball starts rolling I'm hoping there won't be a huge amount of effort required; it's basically 'just' replacing the Neo Geo-specific code with Amiga-specific code! Having said that, there's currently about 1,000 lines of Neo Geo-specific code... though in contrast about 7,000 lines of Xevious core code.

Due to circumstances beyond my control, I've had a little more time on holidays to work on Xevious, though - again - that may not be the case (hopefully) for the remainder of the holidays. Either way, I might take a bit of a break after finishing the transcode before I dive back into it.

Light at the end of the tunnel - and it's bright!

I've found some time - somewhat unexpectedly - to work on Xevious on holidays...

I finally finished the flying enemies getting destroyed, with a doozy of a bug that again, I had to sleep on. Turned out to be an inadvertant re-use of d7 in an inner loop subroutine call.

And that was the last piece of actual gameplay code that remained to be transcoded. Or at least so I thought... turns out during subsequent play testing that the Bacura don't destroy the Solvalou yet! Oops, I'll have to find that!

I've also started on what should be the last remaining piece of 'housekeeping' code - the high score entry screen. Somehow I've managed to introduce a glitch in the in-game high score update which I've never seen before?!? The high score entry isn't complete, and it's not quite right, but probably only a half-hour effort remains.

EDIT: Wait, I haven't explicitly coded or tested a 2P game yet...

Despite all of the above, the transcode is very, very close to being 'finished'. And by that I mean a first-pass, unoptimised transcode. The MAIN and SUB ROM code shouldn't require much in the way of optimisation; it's mainly the Neo Geo code. Other than that, it's a case of going back through the Z80 listing, reviewing the 68K code, and of course a lot of play-testing. There are some bugs, but it's basically completely playable as-is.

No eye-candy today. Aside from high score entry there's not really much new to show anyway. In my next post I should be announcing a major milestone as described above!

UPDATE: Transcode is complete! Buggy, but complete.

UPDATE #2: More work on the high score entry screen. Almost working...

GAME OVER - for 2022 at least!

In the process of transcoding all the enemy AI logic, I ended up finishing off a few other housekeeping routines as the logic was embedded in them. Those routines comprised transition of one Solvalou to the next, and ending the game. Not quite 100% there as there's still no high score entry, but close now.

Game Over!

I also implemented the Solvalou being hit, exploding and ending the turn. A few bugs in the transcode made harder work of it than I'd hoped - including one I had to sleep on - but done and dusted now.

I also discovered that I haven't implemented the tile flip for 2x2 tile sprites completely. As the Solvalou explodes, the same sprites are displayed for a handful of frames and they are periodically flipped based on an asynchronous timer. I haven't tackled this just yet; I'll have to consider the best approach. I may even leave it until I work on optimising the Neo Geo code, as anything further I do now may end up getting re-written anyway.

I had actually looked at the destruction of the flying enemies first, but there was a snag I hadn't expected. It's just a technical detail concerning the difference in implementation between the Z80 and 68K. I just have to work out the optimal solution. Not a big deal, but finishing off the Solvalou seemed more straightforward. A moot point now as I have little option but to tackle it next.

EDIT: I forgot the Bacura! 😳

At a bit over 5,500 lines in the MAIN program, the 2:1 ratio of Z80 to 68K instructions is almost exactly right; I'm really getting down to the last few functions to be transcoded now. Annoying that I've pretty much run out of time to work on it this year, but sometimes you just have to take holidays. 😜

UPDATE: I managed to squeeze a bit more time on the transcode; some more housekeeping so that the high score is updated on-the-fly, and bonus Solvalou awarded according to dipswitch settings.

Bonus Solvalou at 20K and updated High Score

In the process I decided to byte-reverse the storage of all the scores in the game. It made the bonus calculations easier, without affecting the efficiency of any of the other routines.

Finally, I fixed a bug that has been in the transcode and eluding me for weeks now if not longer. It involves the score occasionally appearing as a solid black line, and after implementing high score update, it started occurring at the same time for the high score as well - so doubly annoying! A theory came to me out of the blue tonight - it turned out to be completely wrong, but by then I was determined to find the issue.

It was between the core and the Neo Geo layers; in the core's score display code (a BCD print routine) I was sending the tile code as a byte but the Neo Geo code was assuming the upper byte of the word was zero... which it was... most of the time... until it wasn't. So nice to have finally fixed it!

UPDATE: found a few free minutes to do the Bacura. I had to add support for double-height (only) sprites.

Bacura - the forgotten foe!

Xevious reveals all its secrets.

I've spent the last few sessions trying to finish off the RE and the missing pieces of the AI puzzle.

As a result, the SUB CPU ROM is now 100% complete. All the variables related to the game logic have been understood, named and the routines documented. I could probably spend an hour or so just tidying things up, adding a bit more annotation, but it's essentially done.

Correspondingly, the MAIN CPU ROM is all-but-complete. I haven't actually gone through the disassembly with a fine-tooth comb just yet, but the only missing annotations should be the miscellaneous housekeeping such as score and bonus updates. If I had to guess, I'd say it's 95% complete now.

As for what I've loosely referred to as 'AI' - for want of a better (more concise) term - I believe I've now RE'd all the logic that controls the difficulty of the game. It's not quite as complex as the advertisements suggested back in the day, and I wonder if plenty of other games don't have similar logic built in?!?

At the core of the logic is a couple of tables. First is a table that contains pairs denoting the number of flying enemies and an offset into a 2nd table. That second table is a list of enemy types in (roughly) increasing order of difficulty. So Toroids tend to appear first, with Zoshis and Brag Zakatos appearing later.

The map data contains a 'place holder' object that indexes into the 1st table; for example it may specify 3 Toroids. However there is another variable that is added to the index read from the 1st table, and used for the 2nd table - that variable is what I've named enemy_AI_level. This variable starts out at 0, and is incremented throughout the game - the effect being that the latter index gets larger and larger, causing more potent enemies to appear.

The enemy_AI_level is increased using a few different mechanisms:

• decremented at the start of each new Solvalou according to the STARTING LIVES dipswitch setting
• incremented explicitly in the map data accoring to the DIFFICULTY dipswitch setting
• incremented explicitly in the map data depending on the current score

Incidentally, the enemy_AI_level is decreased by 2 each time a Zolbak is destroyed.

The adjustment depending on the current score is interesting; the amount of adjustment is calculated by dividing the current score (in thousands) by the number of Solvalou that have been used thus far. IOW the average score per Solvalou for the current game. The adjustment factor is (seemingly) equal to the thousands of points scored per Solvalou, with an upper limit of 16, so if you average more than 10,000* points per Solvalou - which is pretty easy for a seasoned player - then the game will play at effective maximum difficulty for that dipswitch setting.

[* Technically the score is represented in BCD, whilst the division operates as if it were in binary. 9,000 points on 1 Solvalou ($09/1) results in an increment of 9, whilst 10,000 points ($10/1) results in an increment of 16. Confusing, but there's still a monotonic increase.]

The other piece of RE I finished today was the Bacura, which operate with their own set of variables, not affected by the enemy_AI_level but rather explicit values in the map data. I'm yet to implement any of the Bacura logic in the transcode; that's next!

Once I finish the Bacura code and debug a few minor issues in the flying enemies, it'll be a case of implementing the destruction of both flying enemies and the Solvalou, and then tidying up all the remaining housekeeping functions. Probably a week or so there, but I'm fast running out of time for the year.

The final stage (before sound) is then optimising the Neo Geo code, and getting it running at 100% on my real AES hardware!

UPDATE: enemy_AI_level is reduced at the start of each Solvalou, not adjusted at the start of each area.

Come fly with me...

I've started on the flying enemies. As a direct result, the SUB CPU transcode is 99.99% complete. There's one small AI-related routine that I'll leave until I've RE'd the logic. Otherwise, the Toroid now appears!

The first AI-driven flying objects - Toroids

As I've mentioned before, the ground-based enemies are not affected by the adaptive AI; every enemy and even their firing frequency is fixed in the game, regardless of the DIFFICULTY dipswitch setting or how well the player is playing. It's the flying enemies that respond to these parameters.

I haven't RE'd all the AI logic just yet, but I do know roughly how the flying enemies are selected.

In addition to all the ground-based objects, the map data contains 'placeholder' objects which invoke different functions for the flying enemies. These include:

• set the number and type of flying enemies explicitly
• increment the AI level (using the DIFFICULTY dipswitch setting) and re-calculate the number and type of flying enemies
• reset the number and type of flying enemies (stops new ones appearing)

The MAIN CPU ROM contains a table of flying enemy numbers and types in order of increasing difficulty; the 2nd function above uses the AI level to index into this table.

Once those variables have been set in the SUB CPU program as the map is being generated, the MAIN CPU program loops over a function that spawns flying enemies according to the current number and enemies type setting.

Obviously I had to implement these functions in order to have the Toroids appear, but I will note that I wasn't sure how/when those variables were being cleared. And when I was debugging the spawning, I actually couldn't find where they were being cleared - this meant that the routine was spawning flying enemies every single invocation - once every VBLANK!?!

So I went back to the arcade code and found exactly the same behaviour! And then it finally dawned on me... the number of flying enemies represents the number AT ANY ONE TIME, not the total number! So for example, 3 Toroids actually means 3 Toroids on-screen, and as soon as one is destroyed or leaves the screen, another one is spawned! FTR spawning an object that already exists has no effect.

I should have realised that a lot sooner, as there are a lot more Toroids than the handful that appear in the map data at the start of Area 1, for example. But I haven't been focusing on flying enemies for quite a number of weeks now.

Now that I understand this, I can proceed with transcoding more flying enemies.

FUN FACT: On the EASY dipswitch setting, the AI level increment used in the above function is set to 0. However it is actually incremented elsewhere, so it does get a little harder.

UPDATE: Toroid, Jara and (1 type of) Zoshi now complete. After some debugging, I'm confident now that the AI is levelling up correctly, as I see the same sequence and number of flying enemies as on the original. I'll have to ramp up the difficulty to HARDEST now to get all the enemies to appear sooner.

In the process I also did some miscellaneous code tidy-up - including the elimination of some duplicated inline code - and added the missing messages at the start of each Solvalou!

Didn't realise these messages were missing!

The MAIN program is around 5,000 lines now. If I'm using the metric that the 68K code is round 1/2 the number of lines of the Z80 code (~11,000 lines) it means I'm getting close to the end now. Aside from finishing off a handful of flying enemy object handlers, there's the destruction of said enemies (common routine), destruction of the Solvalou, and some miscellaneous housekeeping like zeroing variables, updating high scores, awarding bonuses based on score etc. And that's about it, except for sound!

UPDATE #2: I've been flying through the flying enemies and they're all transcoded now except for the Bacura (spinning mirrors) as they are handled in their own routine. There are a few minor bugs but they shouldn't be too difficult to fix. I will need a way of injecting specific enemies into the game at a certain point; that's what build options are for! I also need to document the behaviour of the different variants of certain enemies (Zoshi, Zakato, etc).

I've also done a little bit more of the AI RE in the process; I now know it keeps track of the number of Solvalou you have used in the game, and does some calculation with your score; I'm betting it's working out your average points per Solvalou - but yet to RE any further.