Grim Fandango character is partially rendered on Voodoo3

[hkevinli]

When running Grim Fandango full game, the character is partially rendered after enabling 3D Hardware Acceleration in the game menu.
Guess OS is Windows98 SE. Voodoo3 driver is version 1.07 WHQL and DirectX 7 is installed.

# configuration file generated by Bochs
plugin_ctrl: voodoo=false, unmapped=false, biosdev=false, speaker=true, extfpuirq=false, parallel=false, serial=false, busmouse=false, e1000=false, es1370=false, gameport=false, iodebug=false, ne2k=false, sb16=true, usb_uhci=false, usb_ohci=false, usb_ehci=false, usb_xhci=false
config_interface: win32config
display_library: win32
memory: guest=128, host=128, block_size=128
romimage: file="R:\bochs\BIOS-bochs-latest", address=0x00000000, options=none, flash_data=none
vgaromimage: file="R:\bochs\VGABIOS-lgpl-latest-banshee.bin"
boot: cdrom, disk
floppy_bootsig_check: disabled=0
floppya: type=1_44, 1_44="C:\Users\user\Desktop\Retro_PC\os\Microsoft Windows 98 Second Edition\Windows 98 Second Edition Boot.img", status=ejected, write_protected=0
# no floppyb
ata0: enabled=true, ioaddr1=0x1f0, ioaddr2=0x3f0, irq=14
ata0-master: type=disk, path="c.vhd", mode=vpc, cylinders=0, heads=16, spt=63, sect_size=512, model="Generic HDD", biosdetect=auto, translation=auto
ata0-slave: type=cdrom, path="C:\Users\user\Desktop\Retro_PC\os\Microsoft Windows 98 Second Edition\Windows 98 Second Edition.iso", status=ejected, model="Generic CDROM", biosdetect=auto
ata1: enabled=true, ioaddr1=0x170, ioaddr2=0x370, irq=15
ata1-master: type=disk, path="C:\Users\user\Desktop\Retro_PC\software", mode=vvfat, journal=none, cylinders=41610, heads=16, spt=63, sect_size=512, model="vvfat HDD", biosdetect=auto, translation=auto
ata1-slave: type=none
ata2: enabled=false
ata3: enabled=false
optromimage1: file=none
optromimage2: file=none
optromimage3: file=none
optromimage4: file=none
optramimage1: file=none
optramimage2: file=none
optramimage3: file=none
optramimage4: file=none
pci: enabled=1, chipset=i430fx, slot1=voodoo, slot2=none, slot3=none, slot4=none, slot5=none
vga: extension=voodoo, update_freq=0, realtime=1, ddc=builtin, vbe_memsize=16
cpu: count=1, ips=70000000, model=amd_k6_2_chomper, reset_on_triple_fault=1, cpuid_limit_winnt=0, ignore_bad_msrs=1, mwait_is_nop=0
print_timestamps: enabled=0
debugger_log: R:\bochs\debug.log
magic_break: enabled=1 0x8
port_e9_hack: enabled=false, all_rings=false
iodebug: all_rings=0
private_colormap: enabled=0
clock: sync=both, time0=local, rtc_sync=1
# no cmosimage
log: R:\bochs\log.txt
logprefix: %t%e%d
debug: action=ignore
info: action=report
error: action=report
panic: action=ask
keyboard: type=mf, serial_delay=150, paste_delay=100000, user_shortcut="ctrl-alt-del"
mouse: type=imps2, enabled=false, toggle=ctrl+f10
sound: waveoutdrv=win, waveout=none, waveindrv=win, wavein=none, midioutdrv=win, midiout=none
speaker: enabled=true, mode=sound, volume=15
sb16: enabled=true, midimode=1, wavemode=1, loglevel=0, dmatimer=1000000
voodoo: enabled=true, model=voodoo3
usb_debug: type=none

https://archive.org/details/grimfandangousarerelease

[Vort]

I tested this game with Windows 95 + DirectX 8 and Windows XP + DirectX 9, both cases show incorrect, partial rendering.
With TiledMode disabled in Windows XP, problems disappear.
This led me to thinking that this problem is similar to problem with demo (#462) and caused by game bugs.
However, this time there are few sources suggesting that game should work correctly with Voodoo 3:

1. YouTube video Grim Fandango [Ep.1] - Retro PC Gameplay Walkthrough [3DFX Voodoo3 - Athlon XP 1500+][No Commentary].
2. Comment from PCem repository: Switching to Vodoo Banshee or Voodoo 3 makes all of the above issues go away. The game displays no graphic glitches ....

Which means that bugs, which prevent correct rendering, are most likely in Bochs emulation and not in the game.

[Vort]

A week ago I noticed that Bochs uses simplified approach to handling of tiled surfaces.
It just multiples pitch by 128 and that's it.

Bochs/bochs/iodev/display/voodoo.cc

Lines 581 to 583 in c0f4441

	if (v->banshee.overlay_tiled) {
	pitch *= 128;
	}

But to match real hardware, more complex transformations are needed.
For example, this line

Bochs/bochs/iodev/display/banshee.cc

Line 1604 in c0f4441

offset = (v->fbi.lfb_base + y * pitch + x) & v->fbi.mask;

should be implemented like this (adapted from PCem):
offset = v->fbi.lfb_base + (x & 127) + ((x >> 7) * 128 * 32) + ((y & 31) * 128) + (y >> 5) * pitch * 128 * 32;
or like this (adapted from CSIM):

    int tb = (v->fbi.lfb_base >> 12) & 0x3fff;
    int xb = (v->fbi.lfb_base >> 0) & 0x7f;
    int yb = (v->fbi.lfb_base >> 7) & 0x1f;
    int xl = xb + x;
    int yl = yb + y;
    int xt = xl >> 7;
    int xo = xl & 0x7f;
    int yt = yl >> 5;
    int yo = yl & 0x1f;
    offset = (((tb + yt * pitch + xt) << 12) + (yo << 7) + xo) & v->fbi.mask;

I was hoping that problems with this game can be solved without such change, but looks like there is no other way.
Such change will be complex, will make emulation slower, may introduce regressions, but I think it is needed to be done.
I don't know however if I will be able to make such massive rewrite, maybe it will be easier for @vruppert to do this.

[vruppert]

Such a change must be done in the memory read/write code, all of the 3D code writing to memory and the rendering code. Since @stlintel and me would like to have a new Bochs release in the near future, I'd like to postpone such a rewrite until after that.

[Vort]

Ok, I agree. Let this issue stay open then.