Introduction
SpyHunter is a great example of a reboot done correctly. The 1983 arcade classic got a reboot on the PS2 and other platforms in 2001 and it looked well ahead of its time. One thing that caught my eye earlier this year was that the reboot also featured a Japanese release, which hasn't been dumped. On top of that, I wanted to play this again in stunning 4K. So, time for another media project?
|
|
First, I had to get a dump of the Japanese Release. I found a used copy on
eBay for $15. So I had that delivered and I dumped the disc myself via
dd (Unix utility). After testing on PCSX2, it works just as
well as the USA release. After that, I cranked the internal resolution up
to 3840x2160 and went to work.
This project was also a challenge to see how fast I can make and release content. The challenge was simple. Upload a level every day until (and including) my birthday. June 19th!
There's 14 levels. My birthday is in 14 days inclusively (June 19th). One level of footage per day in 4K 60 FPS where the last level is uploaded on my birthday? Sure. I'm up for a challenge.
— Claıre 🌺 (@iDestyKK) June 6, 2020
There's 14 levels. My birthday is in 14 days inclusively (June 19th). One level of footage per day in 4K 60 FPS where the last level is uploaded on my birthday? Sure. I'm up for a challenge.
— Claıre 🌺 (@iDestyKK) June 6, 2020
Like I said in my tweet, I'm always up for a challenge. So let's get to it.
Setup
For those who don't know, when I do rapid content creation, I usually end up writing a lot of scripts and automation tools to do the job for me. Ironically, this ends up taking a lot of time. But once it's all done, the job becomes as simple as recording gameplay and placing files in the right spots. This is coming from me, someone who effortlessly made a 2000+ video playlist on Modern Warfare 2 matches (and that's just one game I play). So I'm used to doing this.
Patches
The USA release has widescreen and 60 FPS patches available during gameplay. They come with recent releases of PCSX2 and are applied automatically upon game boot. The Japanese release had no patches. So, like usual, I had to take matters into my own hands and make the codes myself. They weren't difficult to create since the game isn't that much different between regions. Here's the codes:
Widescreen Patch [JP]: [GitHub] [PCSX2 Post]
gametitle=Spy Hunter (SLPM-65090)
comment=Widescreen Patch (16:9) by iDestyKK
//FOV
patch=1,EE,20371360,word,3F40000060 FPS Patch [JP]: [GitHub] [PCSX2 Post]
gametitle=Spy Hunter (SLPM-65090)
comment=60 FPS by iDestyKK
//60 FPS
patch=1,EE,202D51F8,word,00000000The results certainly show:
4:3 (2880x2160)
|
16:9 (3840x2160)
|
The 60 FPS code works flawlessly, as expected. But it speeds up the animation of the entering and leaving of the weapons van. I don't intend on this playthrough being a speedrun or going for any kind of record, so this is fine. I don't use the weapons van in any mission except the forced ones in the first and ninth missions anyways.
The HUD is stretched, unfortunately. But, the 3D world drawing and camera are at the appropriate 16:9 aspect ratio. Is it worth the extra resolution? Well, most displays out right now are 16:9. So this gets rid of any black bars that would show on most people's monitors. I think that's worth it.
Theoretical Correct HUD Visualiser
If you were curious, this is what the game should look like if the hud were not stretched. You may use the slider to adjust the width of the screen resolution. Consequently, it'll also adjust the FOV correctly without stretching the HUD. As a bonus, it'll supply the hex value if you wanted to play it at that resolution. Simply adjust the widescreen code above.
|
|
|||||||||||||
|
|
||||||||||||
Honestly, if there were some HUD overrides to make it not stretch, I would've done this in ultrawide. It looks better if you ask me.
Video/Audio Format
As usual, I'm going for a completely overkill setup. This game supports Dolby Pro Logic II, which means it's possible to get surround sound channels out of the matrixed stereo stream that PCSX2 outputs. With the codes above, and the capabilities of PCSX2, 4K 60 FPS is also achievable. So this is what I'm looking at as a reasonable checklist:
- 3840x2160 non-stretched output at 60 FPS
- Lossless 5.1 Surround Sound and Stereo audio tracks (FLAC)
- Chapter Data for each segment in the video
-
SRT Subtitles (
en-us,en-gb, andja-jpif possible) - libx265 with yuv420p10le, encoded at CRF 17
- Watermarked and non-watermarked variations.
Dolby Pro Logic II to 5.0 Surround via FFmpeg
A Dolby Pro Logic II stereo stream containing surround information can be "dematrixed" via FFmpeg:
UNIX> ffmpeg -i "file.wav" -f lavfi -i anullsrc=channel_layout=mono:sample_rate=48000 -filter_complex "[0:a]surround=FL|FR|FC|BL|BR[ma];[ma][1:a]join=inputs=2:channel_layout=5.1[fa]" -map "[fa]" -c:a flac -compression_level 12 "5.0.flac"Here is what a before and after looks like via that command:
Original DPLII Stereo Stream (FL, FR)
|
5.0 Surround Stream via Dematrixing (FL, FR, FC,
|
This command actually creates a 6th channel, being LFE (Low Frequency Effects). Though it's completely silent. From that 5.0 stream, it's possible to make 5.1 by utilising equalisation and inversion to properly extract the LFE from the existing 5 channels. I go over this more down below.
Scripts
These clips were intended to be segmented. By that, I mean the level select, level gameplay, ending cinematic, and results screen were all recorded as separate files, and then concatenated at the end. PCSX2 doesn't have reliable TASing tools to allow a record and replay easily at this time, so everything else had to be done in a straightforward recording.
The encoding and concatenation steps were automated via scripts. As usual, I wrote some stuff to have FFmpeg blast through these files automatically. You may find those in my render_tools GitHub repository. I'll go over these scripts more in a bit. But there needs to be actual footage to encode first. So...
Recording
With PCSX2, recording to lossless AVI and WAV is trivial. Though, I don't get the pleasure of savestating and reloading for a perfect run. So mid-level gameplay must be caught all in one shot. That's ok. That's where the real skill shows. The other segments, like the mission selection, can be recorded separately and merged in post.
Video Segmentation
Parts of the recording can be segmented. The recordings are split (usually) into 4 parts:
- intro.avi - Mission Selection, Overview, Loading
- mission.avi - The actual gameplay...
- end_mov.mkv - Ending Cinematic
- clear.avi - Results screen
- bootup.avi - PS2 Bootup Sequence, Game Load, Intro Clips
- credits.avi - Game credits sequence
- nostra_intro.avi - NOSTRA Introduction Clip
You can clearly tell where some of the segmentation occurs in some of the
missions. For example, sometimes, there's reverb that is cut off right at
the end of a mission, since it switches to end_mov.mkv.
"Clara, why splice the ending clips?"
PCSX2 has an issue playing videos in this game. There's hacks in the settings to mostly fix it. Even with those, it completely breaks after the 60 FPS patch has been applied. Constant flickering, running at the wrong framerates, and even freezing the game. All of these have happened while recording the run. On top of all of that, it stretched the video. I wanted the correct aspect ratio. So, rather than try to fix it via the emulator, I simply extract the video files from the game's ISO, encode those, and concatenate them into the final video. Problem solved.
Gameplay Recording (intro.avi, mission.avi, clear.avi)
These are straightforward. For intro.avi, I simply record the
mission briefing, weapons, and enemies. I stop recording once the loading
screen shows after a level has been selected. For mission.avi,
I record the mission loading screen, and all gameplay. It cuts off just
before the mission ending video plays. clear.avi is simply the
loading part after the ending cinematic, as well as the results screen.
These are all processed and encoded into their MKV variants via
render.sh.
So behold:
UNIX> ./render.sh dir
All this does is go into a directory dir and search for
intro.avi, mission.avi, and
clear.avi. Upon finding them, it'll encode raw and watermarked
versions as MKV files. 6 MKV files created total.
| Original | MKV Raw | MKV Watermarked |
|---|---|---|
| intro.avi | intro.raw.mkv | intro.wm.mkv |
| mission.avi | mission.raw.mkv | mission.wm.mkv |
| clear.avi | clear.raw.mkv | clear.wm.mkv |
Preparing Ending Cinematic (end_mov.mkv)
From my little rant above, end_mov.mkv comes directly from the
game files. It is not recorded from the emulator at all. All of these
cinematic videos have an aspect ratio of 48:25 (1.92). Black bars were
added so they can be in 4:3. Yuck. We can crop them to 16:9 with no loss in
content in the original video, as it's only chopping off black bars. This
crop makes sense, as the gameplay is in 16:9. So having the cinematic as
close to the gameplay as possible looks more natural. Let's do that.
Original 4:3 Black-Bar Master (640x480)
|
Cropped and Upscaled 16:9 Reencoding (3840x2160)
|
And yes, I know upscaling doesn't give it higher quality. But the resolution has to match the other parts for the concatenation to take place. This requires reencoding. So, there's actually some loss in quality. There is no way around this.
First, I extracted the videos from the game ISO. Then I remux'd it via
ffmpeg into an end_mov.mkv. There is no
reencoding going on here. It's the original streams straight from the game.
Now let's prepare it for concatenation. So behold
render_endmov.sh...
UNIX> ./render_endmov.sh dir
It's quite simple. Give it a directory dir and it'll look for
an end_mov.mkv in there. It'll crop it, upscale it to 4K, and
prepare it for concatenation with the other parts. It creates two versions,
one with the watermark (end_mov.wm.mkv), and one without the
watermark (end_mov.raw.mkv).
In the game files, these are stored as PSS files in the
MOVIES directory. Here's the appropriate labelling for each
mission:
| # | Level Name | File Name |
|---|---|---|
| 1 | Test Track License | SPY_PG1.PSS |
| 2 | Dragon Strike | SPY_GE1.PSS |
| 3 | Route Canal | SPY_VE1.PSS |
| 4 | Swamp Venom | SPY_KW1.PSS |
| 5 | Double Vision | SPY_FR1.PSS |
| 6 | Columbian Extract | SPY_PA1.PSS |
| 7 | IES Testing Facility | SPY_PG2.PSS |
| 8 | Escort Service | SPY_ENG.PSS |
| 9 | German Blitz | SPY_GE2.PSS |
| 10 | Terrorist Lock Down | SPY_PA2.PSS |
| 11 | French Kiss | SPY_FR2.PSS |
| 12 | Locked Keys | SPY_KW2.PSS |
| 13 | Venetian Blind | SPY_VE2.PSS |
| 14 | Eye of the Storm |
SPY_ME1.PSS (Lose) SPY_ME2.PSS (Win / Car) SPY_ME2B.mkv (Win / Bike) |
Extract, remux to MKV, run render_endmov.sh on them, done.
Issues while recording
Corrupt Vertices Being Drawn
Occasionally there were triangles that appeared out of nowhere and flickered for around 1 to 2 frames. Corrupt vertex buffer perhaps? A wonderful example of this is in level 8 (Escort Service). Here's an unedited cut where it's at its worst. Even the thumbnail shows it...
The first 5 seconds, I intentionally show that things are fine. Then around the train area, suddenly the screen is covered with broken vertices. Lovely.
Getting around this was simple. Since they only appeared for a frame or two (most of the time), I just duplicated either the previous or the next frame. I'm pretty sure I missed a few. Most are gone though. In the final version of the clip above, you can still see the flickering triangles in the reflection of the car. I guess the reflections don't update every frame.
FOV Adjusting via Rear-View Mirror
Now this one was an annoying one. A lot of times in-game, if enemies appear behind the player's vehicle, the rear-view mirror would automatically open to show the enemies. This was particularly annoying because the 60 FPS + Widescreen cheats completely break when that mirror is open. It flips between 4:3 aspect ratio and 16:9 aspect ratio every other frame.
I can see why it does this, from a game developer perspective. The game projects the world twice. One is the camera behind the car (the main view) and the other is for the rear-view mirror. It isn't possible to just "draw" both simultaneously. It has to project on one camera, switch, project on the other, and overlap the two. So here, it's just failing to set the FOV back to the 16:9 aspect ratio after rendering the rear-view mirror's perspective.
There's two ways around this:
-
Use Cheat Engine, find the exact address of the instruction where
the FOV is being reset, and set it to a
NOP. This address is different every time PCSX2 restarts. So I couldn't make a code for it. This also prevented the mirror from properly flipping. So objects behind but to the right showed on the left instead. Yes, that has to be edited... - Disable the rear view mirror (set it to "Manual" and never use it).
I used both. I think it's obvious which one I went with in the end...
For the first 2 levels (in both the English and Japanese runs), I did patch the instruction out so I can use the mirror. However, when I was in Route Canal, you can explicitly see I wasn't putting up with it anymore.
frame02799.png from Route Canal in the JP playthrough.
Adjusting バックミラー (Back Mirror) to マニュアル (Manual).
By the way, this option resets every time the game resets. It isn't saved. So each time I went to record a level, I had to fix it. Ruined a few potentially good runs. Oh well.
Extra data processing (Audio, Subtitles, Chapter Metadata)
There's more to the perfect footage than video. Especially when delivering a high-quality deliverable to YouTube. Audio is needed. The audio that is currently in the MKVs is unacceptable, as it's a matrixed surround sound stream in stereo. Things sound off. That's no way to have a good listening experience is it? I also want the MKVs to have chapters and subtitles. Let's get to it.
Stereo and 5.1 Audio Track Generation
These MKVs with the .raw.mkv and .wm.mkv discrete
naming get messy really quickly. So they are moved into a
segments directory, which contains a raw and a
watermark directory holding the files renamed from
intro.wm.mkv back to intro.mkv, and etc. The
directory structure is like:
./
segments/
raw/
clear.mkv
end_mov.mkv
intro.mkv
mission.mkv
watermark/
clear.mkv
end_mov.mkv
intro.mkv
mission.mkvNow that they are reorganised, we can optimally generate audio tracks. The audio in all of the AVI files (and all of the MKV files) are the matrixed Dolby Pro Logic II tracks in stereo. So, to get a discrete 5.1 and Stereo Master, they have to be extracted, concatenated, and manually mixed.
I mentioned how to extract a discrete 5.0 stream from Dolby Pro Logic II up
above. With this in mind, I thought ahead, and wrote a script to automate
the initial setup. Behold
gen_concat_audio.sh:
UNIX> ./gen_concat_audio.sh dir
Just like the other scripts, it is given a directory dir,
looks for a segments/raw directory and the video files listed
above. If it finds them, it concatenates them and generates a
5.0.flac file. This can be done by the original AVIs, but it
takes a much longer time to extract from 50 GB AVI files as opposed
to 1-2 GB MKVs.
There's two 5.1 surround sound layouts: Back and Side. The channel layout
for 5.1 (which this 5.0.flac abides by too) here is the back
layout where the final 2 channels are rears. Thus:
| # | Name | Description |
|---|---|---|
| 0 | FL | SPEAKER_FRONT_LEFT |
| 1 | FR | SPEAKER_FRONT_RIGHT |
| 2 | FC | SPEAKER_FRONT_CENTRE |
| 3 | LFE | SPEAKER_LFE |
| 4 | BL | SPEAKER_BACK_LEFT |
| 5 | BR | SPEAKER_BACK_RIGHT |
The stereo track (stereo.flac) can be created right here by
simply mixing the two front and two back channels together. Then mix the
centre into both the left and right. This essentially "flattens" the
surround audio into stereo. You can also have ffmpeg do the
flattening for you. It might do some panning to give you an accurate mix.
The 5.1 track (5.1.flac) is done mostly, but the LFE channel
has to be created. This can be done by importing the stereo track twice.
Convert both to 32-bit float (to allow a sample to go over 1.0 without loss
of detail/clipping). On one, boost the bass frequencies (up to 120 Hz) as
much as possible via equalisation. On the other, invert the waveform. Merge
the two together to cancel out everything but the boosted bass. Then quiet
the track down via amplification (or the reverse...) to an acceptable
level.
These audio tracks will replace the original audio. This is as trivial as muxing it into the MKV with the concatenated video of the 4 MKV segments. A common mistake I see in YouTubers is that they think they have to rerender a video to fix audio. You don't. The streams are separated by nature and can be replaced easily without loss in quality. This goes for any video in any container like MP4, MOV, MKV, etc.
Audio Samples
Curious about the difference? Here's a dplii.flac,
stereo.flac, 5.0.flac, and 5.1.flac
30 second sample from
Level
13 in the US Playthrough (starting at 1:24). I chose this portion
because the back channels are quite active here and I love the railgun
sound effects. Depending on your browser, the surround sound may or may not
work. There are FLAC, AAC (m4a), and OGG variants that all try loading.
|
Dolby Pro Logic II
|
Stereo
|
5.0 Surround
|
5.1 Surround
|
The Dolby Pro Logic II audio is straight from PCSX2. The 5.0 is a direct dematrixing of the Dolby Pro Logic II audio. The 5.1 stream features an LFE channel. I also amplify the channels by some amount (the back channels especially). The stereo is the flattened result of the 5.0 output.
The dematrixed 5.0 and 5.1 audio streams are not perfect surround sound like what you would see today in Blu-ray and modern games. There is lots of crosstalk between the channels. That's a consequence of Dolby Pro Logic II packing 6 discrete channels into 2. It was a "good enough" solution for its time. I guess.
Subtitle Generation (SRT) via Audacity
I want dialogue to be properly subtitled. A quick way to do this is via
Audacity labels. Simply highlight positions in the audio where something is
said and name the label with the words being said. Export it as a
txt file. Then convert it to SRT.
Here's the Audacity labels file for the second level in my US run:
60.785146 62.342060 Mission Route Plotted
83.886927 85.806617 First Objective Complete
145.704977 147.327392 Approaching Weapons Van
181.095798 183.443765 Warning: Missile Inventory Low
222.301113 224.301420 Second Objective Complete
230.745734 232.846812 Third Objective Complete
238.772154 240.409685 Missiles Launcher Empty
241.734825 243.629323 Warning: Machine Gun Ammo Low
243.629323 245.846288 Approaching rendezvous location
246.017598 248.132533 Approaching Weapons Van
248.132533 248.666620 Approaching Weapons Van (lol nope)
265.383380 266.621738 Area ClearedThe format of this file is trivial. It's just "START END TEXT" on each line. Writing a tool for this is trivial. But see, I'm lazy. So there's a tool by some dude on the internet by the username Magicus which does the conversion for you. It's called Audaciter. Simply drag the txt file in and get a valid SRT subtitle file out. Easy.
Video Chapter Information Generation (ffmetadata.txt)
This one is entirely automatic. Simply run
gen_vid_ffmetadata.sh
and it'll grab information from the MKVs (video length) and generate the
chapter information for you. These can be used for YouTube chapters as well
by putting timestamps in the description. I think this is a wonderful way
to allow the user to skip to whatever part of the video they want to see
more easily.
Its syntax is:
UNIX> ./gen_vid_ffmetadata.sh dir title artist
The title field is just the name of the video. I made it the
name of the level I was playing. The artist is "iDestyKK", my
username.
Here
is the link to a ffmetadata.txt that was generated for the
second level in the US playthrough:
Concatenation (finally), a YouTube-ready deliverable
We now have everything needed to generate a YouTube-ready deliverable. The directory structure looks like this:
./
audio/
5.0.flac
5.1.flac
stereo.flac
data/
ffmetadata.txt
segments/
raw/
clear.mkv
end_mov.mkv
intro.mkv
mission.mkv
watermark/
clear.mkv
end_mov.mkv
intro.mkv
mission.mkv
subtitles/
en.srt
en.txt
With these in place, I have one more ace up my sleeve. Just like my
Christmas
Deathmatch Production Procedure post, I have a script specifically for
automating the final step. Behold compile.sh:
UNIX> ./compile.shIt takes no arguments. It just jumps to the level directories, finds all files appropriate for the final MKV deliverable, and muxes the files together losslessly. Two are created. One with a watermark, and one without the watermark.
This final MKV(s) checks all of the boxes mentioned at the very start:
- 3840x2160 non-stretched output at 60 FPS
- Lossless 5.1 Surround Sound and Stereo audio tracks (FLAC)
- Chapter Data for each segment in the video
-
SRT Subtitles (
en-us,en-gb, andja-jpif possible) - libx265 with yuv420p10le, encoded at CRF 17
- Watermarked and non-watermarked variations.
The files are created and uploaded to YouTube. I had to manually readd in the subtitles on YouTube's site, oops. But other than that, you get the amazing 4K video experience this game deserves to have. It looks stunning in this level of quality.
There's another script I wrote,
checklist.sh,
which can be used to tell if a video can be "compiled" successfully.
Obviously, this only takes effect when everything in a row is
"OK". This was satisfying to look at when the project was
completed:
"But YouTube doesn't support surround sound!"
Yes it does. It has been processing my gameplay videos since December 2017 with proper 5.1 audio. YouTube allows files with 2 audio streams (a discrete 5.1 mix and a stereo mix). I'm amazed at the misinformation online about it. You could technically have 2 different audio mixes completely and force your computer and a TV (for example) to have different audio play out, if you wanted. It stores that 5.1 track with all 6 channels properly separated.
Don't believe me? Here's the available streams for one of the videos:
format code extension resolution note
249 webm audio only tiny 53k , opus @ 50k (48000Hz), 2.68MiB
250 webm audio only tiny 69k , opus @ 70k (48000Hz), 3.52MiB
140 m4a audio only tiny 130k , m4a_dash container, mp4a.40.2@128k (44100Hz), 8.09MiB
251 webm audio only tiny 135k , opus @160k (48000Hz), 6.96MiB
256 m4a audio only tiny 196k , m4a_dash container, mp4a.40.5 (24000Hz), 12.19MiB
258 m4a audio only tiny 389k , m4a_dash container, mp4a.40.2 (48000Hz), 24.22MiBBreaking down the streams by number:
-
Streams
249,250, and251areopustracks in Stereo. -
Stream
140is anAAC (LC)track in Stereo. -
Streams
256and258areAAC (LC)tracks in 5.1 Surround.
Just use a YouTube Downloader that lets you specify the audio stream, and
download stream 258 (as it's higher quality than stream
256). You'll see the discrete 6 channels.
"Clara, how much hard drive space did you need for this?"
Plenty.
SpyHunter video project complete 1 day early! The directory storing the lossless 4k 60 fps files is over 2 TB. oh... pic.twitter.com/Lv5Syeb4AY
— Claıre 🌺 (@iDestyKK) June 18, 2020
SpyHunter video project complete 1 day early! The directory storing the lossless 4k 60 fps files is over 2 TB. oh... pic.twitter.com/Lv5Syeb4AY
— Claıre 🌺 (@iDestyKK) June 18, 2020
I plan on keeping the original masters in their uncompressed format (UtVideo). It's one of the reasons I had to upgrade my storage this year. 2.34 TB is much more than double the size of a yearly Christmas Deathmatch project. And I don't even preserve those uncompressed, since I can generate the frames at any resolution and framerate I want. The only reason I am keeping these is because there's no reliable replay support in PCSX2. If there were, I could just keep button inputs and recreate the video at any resolution I want at any time, just like with Christmas Deathmatch. Those files would be kilobytes... It's a shame. I hope they have that working soon.
Video Links & Closing
All 14 levels have been uploaded for both the US and JP releases of SpyHunter. The JP version has been uploaded to both YouTube and ニコニコ動画 (Nico Nico Douga). Here are the links to the playlists:
It feels nice knowing that footage of all 14 levels of the Japanese release are now posted online. I've only been able to find the first 4 or so levels on ニコニコ動画 prior.In addition, I have dumped the game and uploaded its ISO to archive.org. You may find that here:
I'd try a project like this in the future, now that I have a set of scripts that can render and produce videos rapidly. But, I'd really like proper replay support in PCSX2 first. It'd make my life much easier. I initially wanted to record inputs and export each level at 8K. But I had to hold back because playing and recording at 4K is already taxing enough on my framerate. So imagine how I'd do at 8K... That wouldn't be pretty.
Despite my complaints, I'm pretty content with how this project turned out. This game deserves to be seen in 4K given how much effort was put into the visuals. For a game that released in 2001, that's pretty impressive.
