ADFFS 2.81 public beta

[JonAbbott]

2.81 contains some bug fixes for all machines and some new features mentioned below. It is mainly focused on resolving issues when used on a RiscPC's running either RISC OS 3.71 or RISC OS 5.

New features

• *ADFBootFloppy can now be used with HD installed games to run the in-built ADFFS Boot script. This avoids having to extract and maintain the scripts in HD installed games, ensuring they're always using the most recent Boot script. For example replace the James Pond !Run contents with:

  Code: Select all

  ADFBootFloppy F1021101

• !ADFFS will now auto-load itself when opening a JFD file for the first time. This avoids having to run !ADFFS and then open a floppy image
• Whilst a game is running, screen memory is now cache-enabled by default on RiscPC with a StrongARM fitted and cache maintenance is performed before VIDC20's DMA reads the memory. This currently only works for games running under the JIT, but will be extended to games that require VIDC1 emulation
• *MEMA added to be used instead of *MEMORYA and all Boot scripts updated to use it. In addition to Byte, Half-word and Double-word it also support strings, multiple entries on the same command line, skipping to memory locations and cache maintenance
• *HideCommandWindow added to suppress "Press SPACE or click mouse to continue" that some games trigger when exiting back to the Desktop. Note that not all Boot scripts have been updated to make use of it yet - James Pond for example

Changes

• JIT Abort handler wasn't clearing the Device Vector claims on IOMD
• GOARMxJIT wasn't flushing the I-cache
• ADFScreenCaching wasn't included in the IOMD 32bit build and wasn't called by ADFBootFloppy on IOMD 32bit build. It was also corrupting R12 if called by Entry_ADFBootFloppy
• ADFScreenCaching now works on RO 3.71 SA
• JIT report_abort now handles aborts occuring during JIT initialisation
• JIT call_vector_claim was disabling FIQ instead of IRQ on entry and was corrupting R13 on IOMD due to the lack of a NOP after LDMIA R13, {R13}^
• JIT restore_machine_state wasn't restoring the VSync event correctly
• JIT CACHEOP_CLEAN_CODELET wasn't invalidating the I cache on StrongARM
• Added an STR preceding all STM^ on IOMD builds to workaround a StrongARM rev.2 erratum
• JIT call_device_claimant now allocates a transient stack for the vector call, to avoid potential re-entrancy issues
• JIT the following instructions didn't include a NOP after the MSR on IOMD 32bit:
  • <alu>S PC, ...
  • TSTP/TEQP/CMPP/CMNP
  • LDM Rn{!}, {<reg_list>, PC}^
  • MOVS PC, Rn, ...
• JIT core was corrupting the UND stack on IOMD 26bit if the 1st instruction seen was a Copro that needed passing to the previous claimant
• JIT IOMD debug screen output was Black, instead of White
• JIT BL codelet reduced to 9 instructions on 32bit RISC OS
• JFD imager now checks if the disk selection was cancelled, before running the actual imager
• JFD imager disk selection switched to Wimp v3.00 and now defines the menu width
• JIT report_JIT was calling Debugger_Disassemble with SPSR set to the current CPU mode
• EventV handler wasn't claiming the Key event when a task is being terminated
• JIT call_device_claimant now preserves the entry SPSR to avoid SPSR corruption when passing IRQ back to RISC OS on IOMD
• JIT IRQ handler on IOMD was checking IRQ status registers instead of IRQ request registers, causing unnecessary IRQ's to be passed to the OS and game
• JIT IRQ handler on IOMD was corrupting R13 if an IRQ occurred whilst PC was inside a codelet, if there were no IRQ's to pass to the OS
• *MEMA added to reduce Boot script sizes and perform cache ops
• Moved RunType aliases to !Boot and altered so ADFFS will start up if it's not already loaded. This means !ADFFS only needs to be seen during boot and not run to open floppy images
• *HideCommandWindow added to prevent "Press SPACE .." being prompted when a script exits back to the Wimp
• GOARM3JIT now passes to *ADFGO to avoid code duplication
• JIT reset_appspace_L2PT no longer sets page zero access to read/write
• JIT merged ARMop and non-ARMop codepaths and back-ported RO5 ARMop to 26bit builds
• JIT on Low Vector the abort handler wasn't checking for conditional hypercalls, which could cause conditional Page Zero LDR/LDM hypercalls to be overwritten
• added a "Filer_Boot !Boot" to !Run, to ensure its seen by the Wimp, this ensures !ADFFS.!Boot is not rerun by the Wimp when added to the boot tasks
• *ADFBootFloppy will now accept the floppy ID as a value, which will extract the Boot script from !ADFFS.obey and run it
• Added HideCommandwindow to all Boot scripts that use ECHO
• Changed all Boot scripts to use MEMA
• JIT IOC handler was clearing SSBC if the task was paged out
• JIT IOC handler was only processing the first register in an LDM/STM that was touching IOC registers
• JIT IOC handler now emulates IRQB on IOMD
• JIT update_MEMC now only alters the Sound DMA state if its actually changed
• JIT debug via our_XOS_WriteC on IOMD now validates MEMC_Vinit (Rockfall had a bug that pointed the video start into its codespace)
• Writes to MEMC CR now ignore Sound DMA control (fixes sound in Rockfall, No Excuses and Rotor on IOMD)
• JIT wasn't enabling sub-page support on ARM7x0/ARM7500/SA
• ADFRemapVideoMemory wasn't flusing the TLB if it unmapped the MEMC section in L1PT on IOMD 26bit
• JIT TSTP/TEQP/CMPP/CMNP are now left as-is on IOMD 26bit
• Data Abort handler could have corrupt R4,R7 when passing aborts to the OS early if Instruction tracing was enabled
• JIT Page Zero abort detection was checking for <&100000 not <&4000
• JIT on IOMD 26bit the Abort handler now always sets R13 on entry as the OS Abort handler performs an LDR R13
• JIT on IOMD 26bit the UND32/ABT32 stacks are now setup during Module startup, instead of JIT startup and the UND32 stack is left imortal when ADFFS exits. The UND32 stack is only created if FPUEmulator hasn't setup a valid stack
• JIT IOMD 26bit Page Zero abort handler wasn't setting up the word aligned address when proxying writes by the OS
• JIT CACHEOP_CLEAN_APPSPACE_TO_HERE was cleaning 1 instruction too many
• JIT wasn't tracking the number of instructions processed, so could exceed JIT_max_instructions (currently 128) if it didn't exit early due to a branch
• JIT CACHEOP_CLEAN_D_INVALIDATE_I was cleaning 1 extra cache line if the last instruction was the last in the cache line
• JIT IRQ order corrected to IRQ, IRQ1v, Device Vectors
• MEMC Vinit wasn't testing for Vinit=Vend correctly when setting VIDINITA on IOMD
• IOMD if a StrongARM is present, cache flushes are performed to ensure the cached video memory is flushed:
  • On entry to OS_Byte 113
  • On entry to OS_Word 22
  • Before writing to MEMC Vstart
  • If none of the above occurred, the IRQ vector flushes before exit if VSync was active on entry
• "Boot Floppy" no longer forces Screen Caching off on StrongARM
• JIT Branch thru Zero handler could crash if entered in USR without a valid stack
• ADFJoystickKeys wasn't releasing it's ticker correctly when disabled
• SWI handler quick-check now using a slightly improved routine
• LCDGameModes updated to set LCDGM$Status when it is enabled and Boot scripts revised accordingly
• Added volume control keys: CTRL-SHIFT-Up / CTRL-SHIFT-Down in preparation for a switch to 16bit audio (2.81n)
• *JITBREAK wasn't resetting the instruction at the breakpoint to ensure the JIT is entered
• JIT now performs an OS_DelinkApplicaton when shutting down as a fail-safe
• *ADFRemapVideoMemory and *ScreenSize now add a pause after changing screen mode, before changing DA2's size, to workaround a bug in Red Squirrel that crashes the emulator
• JIT now restores the OS_Byte 114 state when shutdown (fixes Inferno's display issues when run after CTRL-SHIFT-F12 terminating Interdictor)
• JIT MRC CP15,0,Rx,C0,C? might return an ARMv3 response if the JIT was in ARMv4 mode and the cache was disabled
• JIT add a check for MRC CP15,0,Rx,C2,C? when running in ARMv4 mode
• JIT STR PC,[PC, ... was corrupting the register used for tmp2
• JIT now ensures its emulating an appropriate OS for the CPU mode. ARMv3 will drop to RO3.11 and ARMv4 will switch to RO3.71. When shutting down it will reset the emulation to the current OS
• JIT OS_SynchroniseCodeArea 0 no longer flushes the full JIT cache. It now cleans the current WimpSlot and allocated region of the RMA. All codelets that aren't immortal are cleared
• JIT updated APCS-R Clib to 6.14 as most OS_SynchroniseCodeAreas 0 have been replaced with ranged syncs
• JIT ARMv4 cache consistency checker wasn't checking MOV PC,PC correctly (Burn'Out [SA] intro @ B8B0)
• JIT environment_handlers was not marking Environment handler codelets as immortal (fixes Burn'Out [SA] crash when run under the ARMv4 JIT)
• JIT when the ARMv4 JIT is active all JITMEMORYA are now made persistent to cover OS_SynchroniseCodeAreas clearing the JIT code cache (fixes Burn'Out [SA]) (2.81rc1)
• *ADFCRC added to allow memory to be CRC checked before patching
• Modified the APD decoder to retain R12 as a pointer to VARS and adjusted all VARS references to be relative to R12
• Removed the embeded MFM CRC table from the 1772 emulator, its now calculated when an APD is loaded
• APD decoder no longer uses a hardcoded buffer limit, its now extended as required (fixes aborts when loading 1.6MB APD's)
• Added CRC checks to boot scripts that modify game code (up to F1007201)
• Alias$RunType_FCE / FE3 modified to support longer file paths
• Added CRC checks to boot script that modify code (2.81rc2)
• *ADFOpen might have trigger an error trying to open a Filer window if it had already reported a Wimp error trying to open the file
• ADFFS$CRC now set when ADFFS is loaded, to avoid errors in Boot scripts that check file CRC's

Known issues

• CTRL-SHIFT-F12 will not terminate the following games, which run natively (this is the ARMv5+ list, many more games run native on IOMD):
  • Botkiller
  • Exodus
  • Freddy's Folly
  • Giant Killer
  • Gloop (from Logic Mania)
  • MicroDrive 32M
  • Minotaur
  • SunBurst
  • TwinWorld
  • Wolfenstein 3D
  • Zool - Ninja of the 'Nth' Dimension
• Games that alter the Wimp Mode might cause a lock when exiting back to the Wimp. This appears to be an external issue when the Wimp can't handle the MODE. Titles need patching to not mess with the Wimp:
  • Cyborg
  • Dreadnoughts
  • Guile
  • Put It!
  • Wizard Apprentice
• WIMP based titles may lock the machine:
  • Abuse
  • Alderbaran
  • Black Angel
  • Demon's Lair
  • E-Type II
  • Elite
  • Emotions - Search for Humanity
  • EuroBlaster (from Flying High)
  • Fervour
  • Foray
  • Frak!
  • Global Effect
  • Hamsters
  • Haunted House
  • High Risc Racing
  • Ixion
  • Joust (from Flying High)
  • Karma - The Flight Trainer
  • K.V.
  • Mah-Jong Patience
  • Merp (from Mirror Image and Merp)
  • Premier Manager
  • Scrabble
  • Shuggy
  • SimCity
  • TANKS
  • The Chaos Engine
  • Virtual Golf
  • WaveLength
• Dominate - has corruption on the wipeout screen, which is corruption on the original floppy. Need to source another copy to image
• Formula Fun - doesn't hide the text cursor and doesn't wait on the track map screen
• Frak! - music is glitchy and the game has a screensaver, which you can't get out of without hitting ESC
• Hostages - Music might be wrong. Check against a physical RO2, ARM2 based machine
• James Pond 2: Robocod (original release) - music key is wrong. Also happens on physical, might be related to a RISC OS update or Module version
• Caverns does not scroll correctly on a RiscPC with VRAM fitted
• Caverns does not display the high-score/ship on a RiscPC running RISC OS 5
• Rotor does not work on a RiscPC
• Games that rely on specific video timings or geometry to palette swap or set the Pointer position may not be correct on IOC with LCDGameModes enabled or IOMD based machines
• Sound is currently very jittery on a RiscPC with a StrongARM fitted, when the JIT is performing cache maintenance. For example when a game starts up or is using self-modifying code
• Using CTRL-SHIFT-F12 to terminate a game on IOMD can result in the Desktop inadvertently shutting down
• The following games crash when installed to long paths:
  • Abuse
  • Wonderland
• Aldebaran hard-locks the system once loaded when running under the JIT

Package testing
Game packages have yet to be updated to support the changes in ADFFS 2.81


Fixed issues

• Boogie Buggy script has an uncommented comment, which generates an error during loading
• Boogie Buggy crashes if you are killed by a little green bug in Mudlark Bog
• Burn 'Out [SA} - hangs on the intro screen if the ARMv4 JIT is used
• Inferno doesn't display correctly if run after Interdictor is terminated via CTRL-SHIFT-F12
• Lotus Turbo Challenge 2 Boot script is generating a Buffer Overflow error
• Burn 'Out [SA} - crashes when selecting a race if the ARMv4 JIT is used
• Boogie Buggy script does not check for, or patch variants. This can result in a blank screen when entering certain levels

[eddebaby]

Jon, the ADFS module appears to be missing from the beta you posted above (adffs281m.zip).

[JonAbbott]

Apologies, I've just added it.

[IanJeffray]

[*]Burn 'Out [SA} - hangs on the intro screen if the SA JIT is used - the game code isn't fully StrongARM compatible

Can you elaborate on that? Given that it works on all my SARPCs here (K, S, T variants, RO 3.7, 4.x, 6.x) ... what aspect isn't "fully" SA compatible?

[*]Inferno doesn't display correctly if run after Interdictor is terminated via CTRL-SHIFT-F12

"correctly" ? Can you give any more detail? Is Interdictor messing up the system or... ? Is only Inferno affected?

[JonAbbott]

Burn 'Out [SA} - hangs on the intro screen if the SA JIT is used - the game code isn't fully StrongARM compatible

Can you elaborate on that? Given that it works on all my SARPCs here (K, S, T variants, RO 3.7, 4.x, 6.x) ... what aspect isn't "fully" SA compatible?

A typo, which I've now corrected. It should say "not fully StrongARM JIT compatible"

I've not looked at the BurnOut Module code to see what the issue is yet as running under the ARMv3 JIT works, but I suspect its either missing an OS_SynchroniseCodeAreas somewhere or is calling OS_SynchroniseCodeAreas with R0=0 to perform a full memory clean. Synchronising the whole of memory is incredibly expensive for a JIT, so it's ignored and only cleans the CPU cache. If it's the later, I can modify the Module code to perform a ranged clean.

There is one issue I'm currently fixing, but you won't notice it on a legacy OS as it fails silently - hence why I've not bothered telling you about it:

• BurnOut_LoadPacked exits with V set when it returns with R0=-1

Inferno doesn't display correctly if run after Interdictor is terminated via CTRL-SHIFT-F12

"correctly" ? Can you give any more detail? Is Interdictor messing up the system or... ? Is only Inferno affected?

This is an issue for me to resolve in ADFFS, something isn't being reset correctly when Interdictor is terminated, which causes Inferno to do odd things like show corrupt graphics on the menu screen or just not display the menu.

[JonAbbott]

Burn 'Out [SA} - hangs on the intro screen if the SA JIT is used - the game code isn't fully StrongARM JIT compatible

This is proving tricky to track down. The BurnOut module does make use of OS_SynchroniseCodeAreas 0 in a few places, but having checked the JIT code, it does flush the whole JIT cache correctly. As that is incredibly expensive (writing around 13.5MB of memory) I've taken the opportunity to recode it to only flush the current WimpSlot and used RMA space. For the codelet flush it now doesn't walk the JIT cache looking for codelets, but assumes they're all now redundant and just resets the codelet heap. I do need to double check the change doesn't impact any of the other games using the ARMv4 JIT (and update the list of games using ARMv4 in the boot script while I'm at it.)

The differences between the ARMv3 and ARMv4 JIT are subtle:

• STM with PC in the list stores PC+8 instead of PC+12
• STR PC stores PC+8 instead of PC+12
• self-modifying code detection is turned off
• MCR/MRC CP15 emulate StrongARM responses
• cache maintenance is left to OS_SynchronisedCodeAreas
• OS emulation is switched to RISC OS 3.71 as a minimum

I've tried systematically reversing all those changes except the self-modifying code detection and it always attempts to jump to &70xx somewhere in the Intro screen that displays the machine spec. The jump appears to be an exit from a random BurnOut SWI, which would imply either stack corruption or an issue in the 26-32bit exit handler.

I've triple checked all the code above for errors, which did turn up an issue when emulating STR PC,[PC... which I've now fixed, but it all looks good otherwise. Systematically turning the changes off would have flagged any further issues.

I suspect it's a hidden bug in the JIT somewhere or failing that, an area of memory that isn't being cleaned by an OS_SynchroniseCodeAreas. I'm going to have to get lucky to find root cause as I've run of out of things to test.

Inferno doesn't display correctly if run after Interdictor is terminated via CTRL-SHIFT-F12

This turned out to by the state of OS_Byte 114. Interdictor was switching the next MODE change to use the banked screen and as Inferno wasn't explicitly changing screen banks, the menu was appearing on the wrong screen buffer. The JIT now restores the state of OS_Byte 114 when shut down to resolve.

[JonAbbott]

Burn 'Out [SA} - hangs on the intro screen if the SA JIT is used - the game code isn't fully StrongARM JIT compatible

I got lucky and eventually found root cause. It was nothing to do with the ARMv4 JIT, but Environment handlers.

Enviroment handlers require a wrapper if the previous handler in the chain is in 32bit space. As there's nowhere to put the wrapper they're put into a codelet (in hindsight they probably need their own dedicated region of 26bit memory.) Where I recoded OS_SynchroniseCodeAreas to assume all codelets are redundant on an OS_SynchroniseCodeAreas 0, I broke them even further, so it now walks the codelet tree and removes all codelets except Environment handlers.

The original issue was two fold. The code that creates the Environment handler wrapper codelet wasn't marking it as immortal, that caused the code that resets a ranged memory block to remove it as the instruction address (which is checked when memory is reset) was pointing at the environment handler entry address. If an OS_SynchroniseCodeAreas was subsequently performed over the address range of the Environment handler, it deleted the codelet so this issue was also affecting the ARMv3 JIT. The reason it wasn't showing up for Burn'Out SA under the ARMv3 JIT was down to the fact OS_SynchroniseCodeAreas 0 is ignored when in ARMv3 mode - the self-modifying code detection handles synchronising memory when code is overwritten.

However, I'm now seeing lots of Page Zero accesses in Burn'Out that need investigating and C raises an Uncaught trap in mymalloc() starting a race so more issues to resolve.

[JonAbbott]

Uncaught trap in mymalloc() starting a race

This is being caused by:

• BurnOut_LoadPacked exits with V set when it returns with R0=-1

When OS_SynchronisedCodeAreas is called, all the JIT'd code over the area in question is removed and of course it then loses any fixes that have already been applied. In the case of BurnOut_LoadPacked it changes the exit to not set V.

OS_SynchroniseCodeAreas really needs to track code fixes and leave them in place. In the meantime I'll probably just change the ARMv4 JIT to ensure all code fixes are persistent. I'll also need to ensure OS_SynchroniseCodeAreas is completely ignored when the ARMv3 JIT is active.

[JonAbbott]

2.81rc1 in the OP. I've started adding file CRC checks to boot scripts that modify game code. If the file being patched doesn't match what's expected, it will report an error - this is primarily to catch alternative releases of games that might crash or hang when patched.

[JonAbbott]

2.81rc2 available in the OP.

Many Boot scripts that patch games now perform CRC checks to ensure self-created JFD match the version of the game the scripts were created for.