Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

pneumatic wrote:

If you want a noisier but higher resolution image, go for TIVTC (I'm then adding a bit of sharpening through my media player's renderer).

Thanks so much. Strangely, my use of the QTGMC filter alone isn't removing the jagged edges. I think I'm doing something wrong in Hybrid.

But TIVTC is what I'd prefer: I want all the noise because that's the raw material for a Topaz AI upscale. I'm also going to follow your lead and use CPU encoding so that I can set the tuner to "grain" to get whatever little film grain texture is in these files. I've set the encode to run a "very slow" two pass job, so it's going to take 12 - 13 hours to process episodes 1.02 - 1.09. Hopefully, these Universal DVD files will become in the best possible shape they can be for upscaling afterwards.

662 (edited by pneumatic 2023-04-09 22:46:51)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

To save time you can make yourself a short clip to experiment on:

ffmpeg -ss 00:00:00 -to 00:00:30 -i "C:\S01E04.mkv" -c:v copy -c:a copy "C:\ShortClip.mkv"

This will output the first 30 seconds of S01E04.mkv to ShortClip.mkv without any transcoding (MediaInfo should report the video track is still MPEG2 720x480 29.97fps interlaced - exactly what was on the disc).

Then if you install Avisynth+ 64-bit you can perform detelecine on ShortClip.mkv like so:

ffmpeg -color_range 1 -colorspace 6 -color_primaries 6 -i "C:\MyAvisynthScript.avs" -c:v libx264 -tune film -b:v 8000000 -pix_fmt yuv420p -color_range 1 -colorspace 6 -color_primaries 6 "C:\Detelecined.mkv"

This will output Detelecined.mkv at x264 720x540 23.976fps progressive 8mbps (confirm with MediaInfo).

Inside MyAvisynthScript.avs put:

clip = "C:\ShortClip.mkv"
video = LWLibavVideoSource(clip, stream_index=-1, repeat=true, cache=false)
audio = LWLibavAudioSource(clip, stream_index=1, cache=false)
AudioDub(video, audio)

TFM(mode=0, slow=2, scthresh=100, PP=3, cthresh=9, MI=128,
\    clip2=PropDelete("_FieldBased").bwdif(field=-1, thr=2, edeint=nnedi3(field=-1)))

TDecimate(mode=0, cycleR=1, cycle=5, hybrid=1, viddetect=2, vidthresh=4.5, 
\    denoise=true, chroma=false, hint=false)

LanczosResize(720,540)  

The above Avisynth script uses filters that don't come with Avisynth+ so you'll have to download them and place their dll files in typically C:\Program Files (x86)\AviSynth+\plugins64.  I've put them all in a zip file here (edit: this doesn't include the HD colourimetry conversion and QTGMC dlls - if you want those too I've put them all here). 

Alternatively here are the author's download pages: [1][2][3][4][5][6] (be sure to get the 64-bit versions).

ffmpeg 64bit can be downloaded here.  Just unzip the files to anywhere, open a windows command prompt, navigate to the \bin folder where ffmpeg.exe is and you're ready to start typing ffmpeg commands in command prompt.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

I've transcoded episodes 102 - 109 with TIVTC and QTGMC. I had a look at them earlier and pneumatic's recommendations have yielded much-improved files. The video quality has gone from severely aliased (DVD) or blurry (Handbrake-detelecined-DVD) to... soft. Soft focus, but watchable and adequate.

My suspicion is that this is probably the upper limit of video files sourced from late-1994 to early-1995 era analog videotape. Season 2's digital videotape episodes, while having a softness to them, were also covered in blocks and grain that an AI upscale could mine to rebuild a facsimile of the original film image.

With the Season 1 episodes, pneumatic's tips have avoided the blurriness that Handbrake created in decombing the files, but I don't see the Season 2 level of grain or the Season 3 level of sharpness needed to bring it to AI-generated 720p. The image is too soft, although better than what's on the disc.

However, I've still loaded everything into Topaz AI for a 720p upscale just to see what happens. My soul won't rest until we try. My expectation, however, is that it'd be best to just put the files in Topaz AI for 540p output so as to add a bit of AI sharpening and film grain but not add more pixels.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

I've had a look at "Fever" in progress of the AI upscale to 720p and... I do not like what I am seeing. The TIVTC + QTGMC transcode had its flaws like how the skin above the actors lips looks white instead of skin coloured. In the SD transcode file, the softness of the image means this flaw blends into the picture and at living room distance, it's a non-issue. But in the 720p upscale, that white line is enhanced and sharpened and you can't unsee it or look away from it.

I think pneumatic just getting a decent image out of these poorly handled DVD files for 102 - 109 was a huge achievement. I think I'll take until tonight to decide if I'm going to continue with this upscale or whether or not I'll just see about Topaz AI sharpening them up as 540p files (and hope that doesn't add more image flaws while offering few gains).

665 (edited by RussianCabbie_Lotteryfan 2023-03-26 12:26:00)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

ireactions wrote:

I've had a look at "Fever" in progress of the AI upscale to 720p and... I do not like what I am seeing. The TIVTC + QTGMC transcode had its flaws like how the skin above the actors lips looks white instead of skin coloured. In the SD transcode file, the softness of the image means this flaw blends into the picture and at living room distance, it's a non-issue. But in the 720p upscale, that white line is enhanced and sharpened and you can't unsee it or look away from it.

I think pneumatic just getting a decent image out of these poorly handled DVD files for 102 - 109 was a huge achievement. I think I'll take until tonight to decide if I'm going to continue with this upscale or whether or not I'll just see about Topaz AI sharpening them up as 540p files (and hope that doesn't add more image flaws while offering few gains).

It would be interesting to see whether you think the original topaz upscaled files is inferior to Pneumatic's method with topaz sharpen and no upscale.

Also, while not likely, is it possible that a different topaz upscale algorithm would provide better results on pneumatic's method than was previously used?

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

I'm seeing the same issues even with 540p AI upscales. I don't know what AI algorithm is really right.

AI upscaling from SD to HD is operates best when analyzing the film grain and extrapolating from that to recreate an approximation of the original film image. That's how a fuzzy but grainy SD image can look great in AI-HD. It's is why Season 2 in its blocky graininess looks very good after an AI upscale. This is why the very grainy Seasons 4 - 5, even in hypercompressed Mill Creek discs, looks terrific after an AI upscale.

Season 3 doesn't really look that much better (and no worse); there isn't that much grain in the SD image, but there's enough SD detail for AI to sharpen, so a good image in SD looks good in AI-HD.

With Season 1 after the pilot, there's a lack of detail, some artifacts from the detelecining/deinterlacing, and a lack of film grain. The grain just didn't survive the film-to-analog videotape transfer. Without grain, AI will just inflate the existing lack of detail and artifacts with more pixel contrast and create bizarre and distracting anomalies.

I think I'm going to run through pneumatic's processing advice again, but this time output the file to 1080p with Lanczos scaling just to see how it looks.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

ireactions wrote:

I'm seeing the same issues even with 540p AI upscales. I don't know what AI algorithm is really right.

AI upscaling from SD to HD is operates best when analyzing the film grain and extrapolating from that to recreate an approximation of the original film image. That's how a fuzzy but grainy SD image can look great in AI-HD. It's is why Season 2 in its blocky graininess looks very good after an AI upscale. This is why the very grainy Seasons 4 - 5, even in hypercompressed Mill Creek discs, looks terrific after an AI upscale.

Season 3 doesn't really look that much better (and no worse); there isn't that much grain in the SD image, but there's enough SD detail for AI to sharpen, so a good image in SD looks good in AI-HD.

With Season 1 after the pilot, there's a lack of detail, some artifacts from the detelecining/deinterlacing, and a lack of film grain. The grain just didn't survive the film-to-analog videotape transfer. Without grain, AI will just inflate the existing lack of detail and artifacts with more pixel contrast and create bizarre and distracting anomalies.

I think I'm going to run through pneumatic's processing advice again, but this time output the file to 1080p with Lanczos scaling just to see how it looks.

In a perfect world, perhaps an aglorithm could add grain back into the image based on lighting and assumed contouring of image.  And then you use the added grain to upscale.  Of course, perhaps an upscaling algorithm could guess upscaling technique based on assumed light/contour shaping as well.   I guess something like that doesn't exist now, because this sort of content (flat without grain) is a smaller use-case for the upscaling marketplace.

668 (edited by RussianCabbie_Lotteryfan 2023-03-26 13:34:47)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

ha, from 2017

http://www.ipol.im/pub/art/2017/192/

http://www.ipol.im/pub/art/2017/192/Fil … POL.tar.gz <-- code
https://ipolcore.ipol.im/demo/clientApp … tml?id=192    <-- demo

I wonder if this would apply at all to content like sliders 1, or if it doesn't have the baseline the algorithm depends on.

Edit:  Not too impressed with the demo.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

I'm just happy that pneumatic's experimentation and explorations have yielded something I didn't think we could have: an acceptable DVD image from the Season 1 Universal DVDs.

Yes, the image still looks like a standard definition analog videotape from 1994 because that's what it is. But it's clear of jagged edges, it isn't obnoxiously distracting me with interlacing lines flashing across the screen, and at living room distance on an HDTV, it looks like an okay DVD.

I don't even think my new transcode to 1080p (with 3GB files per episode) is really even necessary; it just seems better to have the file brought to 1080p with a pixelation-reducing rescaler (Lanczos) rather than my TV doing a linear stretch.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

I watched, on my HDTV, some of what I'm going to call the pneumatic edition of "Prince of Wails" with Lanczos scaling it to 1080p. I compared it to the 540p pneumatic version (stretched to fill the TV) and... the Lanczos version has a slight edge. It's a little smoother and sharper where the 540p version has a faint blockiness (because of the stretch). I'm not sure the Lanczos version needs to be 3GB, however. I think setting the bitrate at 10 Mbps was too much.

It looks like an okay DVD whereas before pneumatic arrived on the scene, it looked like a faded VHS copy of a VHS copy.

Also, as pneumatic noted: the 24 fps frame rate makes the spiral of Earths in the opening titles appear to skip several frames. I'm going to see if I can switch it to 30 fps in Hybrid.

671 (edited by pneumatic 2023-03-27 06:39:19)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

ireactions wrote:

it just seems better to have the file brought to 1080p with a pixelation-reducing rescaler (Lanczos) rather than my TV doing a linear stretch.

Yeah, it's probably better for us to produce 1080p files as it takes out of the equation the unknown upscaling quality of the end user's media player or TV, which could potentially be poor.   

These are the scalers included with Avisynth: http://avisynth.nl/index.php/Resize

The most popular ones seem to be BicubicResize, Spline36Resize and LanczosResize.  They can be tuned for sharpness:

BicubicResize(1440, 1080, 0.0, 0.75) means upscale to 1440x1080 with blur=0.0, sharpen=0.75

LanczosResize(1440, 1080, taps=4) is sharper than LanczosResize(1440, 1080, taps=3), the latter of which appears very close to Spline36Resize.

There is also this Sharpen() filter included with Avisynth: http://avisynth.nl/index.php/Sharpen

There are a bunch of third party sharpeners here http://avisynth.nl/index.php/External_f … Sharpeners which I'm sure can do a much better job than Sharpen(), but I haven't delved into them as I do my sharpening through the media player (MadVR).

If we upscale to HD we must remember to convert the colours to HD otherwise they will be quite off:
https://i.lensdump.com/i/TolEnr.png
(animated .png)

In Avisynth...

# convert colourimetry from NTSC to HD 
z_ConvertFormat(                        
\ colorspace_op="601:601:170m:full=>709:709:709:full",
\ resample_filter="spline36",
\ dither_type="ordered",
\ interlaced=true )

AI upscaling:

nnedi3_rpow2(rfactor=4, nns=1, nsize=0, cshift="Spline36Resize") # AI upscale 4x (720x480p -> 2880x1920p)
Spline36Resize(1440, 1080)                                       # downscale to 1080p

Preview:

QTGMC + AI upscaling (1080p60).mkv
TIVTC + AI upscaling (1080p24).mkv

672 (edited by RussianCabbie_Lotteryfan 2023-03-27 07:32:47)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

@pneumatic  that's all great info, especially on the colors.

Just some anecdotal information:  my brother once used AVI synth scripts to upscale old compressed DVD images of a documentary he created back in 2006.  The image was originally shot in hd on digital but then ripped off of old dvd files he had from the final produced product.     He spent a lot of time getting the script correct for the settings, to find optimal settings.  This was around 2015 I believe.

It did upscale to look pretty good.  BUT, ireactions later used topaz to upscale the documentary for us with Topaz (artemis dehalo algorithm) and it was substantially better.     And things looked so gorgeous, he got a license to topaz and did it in the gaia algorithm, which he showed the end result to a tv producer, who could not even tell the footage was shot in 2003 and resuscitated from old dvd files!    So since you are really into this stuff, I would just suggest you look more at Topaz as well.   I could be wrong but I am not sure avi synth is as powerful of a tool for certain use cases.

673 (edited by pneumatic 2023-03-27 09:39:18)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

RussianCabbie_Lotteryfan wrote:

It did upscale to look pretty good.  BUT, ireactions later used topaz to upscale the documentary for us with Topaz (artemis dehalo algorithm) and it was substantially better.     And things looked so gorgeous, he got a license to topaz and did it in the gaia algorithm, which he showed the end result to a tv producer, who could not even tell the footage was shot in 2003 and resuscitated from old dvd files!    So since you are really into this stuff, I would just suggest you look more at Topaz as well.   I could be wrong but I am not sure avi synth is as powerful of a tool for certain use cases.

I'm sure Topaz can do a better job than nnedi3, I just don't have $300 USD to spare right now.  Although I see they have a free trial version so I'll check it out for sure.   

edit: unfortunately I cannot use it as it's Windows 10+ only.

674 (edited by RussianCabbie_Lotteryfan 2023-03-27 10:32:05)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

pneumatic wrote:
RussianCabbie_Lotteryfan wrote:

It did upscale to look pretty good.  BUT, ireactions later used topaz to upscale the documentary for us with Topaz (artemis dehalo algorithm) and it was substantially better.     And things looked so gorgeous, he got a license to topaz and did it in the gaia algorithm, which he showed the end result to a tv producer, who could not even tell the footage was shot in 2003 and resuscitated from old dvd files!    So since you are really into this stuff, I would just suggest you look more at Topaz as well.   I could be wrong but I am not sure avi synth is as powerful of a tool for certain use cases.

I'm sure Topaz can do a better job than nnedi3, I just don't have $300 USD to spare right now.  Although I see they have a free trial version so I'll check it out for sure.   

edit: unfortunately I cannot use it as it's Windows 10+ only.


I am on Windows 8, so I can relate.  They are dropping a lot of browser support for it even.

I didn't realize Topaz was $300, my brother got it on sale for $160 (still very pricey of course).   It would be nice if they offered one-time upscales based on the length of content but maybe it doesn't fit their business model.

675 (edited by ireactions 2023-03-27 16:45:37)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

I'm getting some moire patterns where there were previously jagged edges. This screenshot from "Eggheads":

https://i.ibb.co/Kq01GDp/Sliders-107-Eggheads-mp4-snapshot-26-09-757.png

Any recommendations from pneumatic for what addresses this? Or is it best left alone?

*

On the subject of Topaz AI -- I think it's a little overhyped. The upshot of Topaz AI and all the experiments in this thread is this: if a video looks good in standard definition at 480i, Topaz AI can make it look good in high definition at 1080p. If it's sharp in SD, Topaz AI can make it sharp in SD. If it's blurry but grainy in SD, Topaz AI can resolve that grain into HD detail.

This isn't actually that meaningful, useful or worthwhile.

If you have a video that looks good at 480i, you can likely upscale it in less taxing ways than $300 upscaling software and still get decent results; if you produced a video that looks good at 480i, you likely have the original materials to produce a high definition version. Why would you need Topaz AI if you could go back to your original film or HD digital masters?

Meanwhile, Topaz AI can't seem to do very much with an SD image that is blurry and lacking in film grain.

AI upscaling could be highly effective in specific use cases that I think are incredibly insular and a bit silly. FOX lost a ton of film and effects shots for THE X-FILES, and a lot of their HD release of the show has used upscaled videotape where the original film could not be rescanned. BABYLON 5 and LOIS AND CLARK had film rescans for HD releases, but the effects were pulled from the standard definition master tapes.

An AI upscale could have helped out a lot there. AI upscaling might be good for a few shots here and there or for 90s effects work that was always a bit painterly. BABYLON 5 effects looked like video game graphics. AI could increase the size of those without degradation.

And sure, if you're an obsessive fan who really wants to watch "As Time Goes By" in 1080p (which I am) and an on-the-fly upconvert isn't good enough for you (and I wouldn't mind better), then yes, Topaz AI could make your dreams come true with that episode, one that was edited on 1995-era digital videotape that preserved the film grain for the AI to reconstruct. And if you're really not happy with your Mill Creek DVDs for Seasons 4 - 5, then Topaz AI can make that grainy image look HD even from those overcompressed discs because of the 16mm film grain.

And yes, RussianCabbie's brother got his footage back, but it was produced in HD in the first place and transferred to a format where that the HD elements were downscaled but not lost.

In the case of episodes 102 - 109 for SLIDERS, they were edited on analog videotape. This unfortunately creates severe generational loss. And for analog videotape, Topaz AI has been pretty useless. Topaz AI can't rebuild what isn't already there either as standard definition detail or film grain, and analog videotape clearly has neither.

Topaz AI is a bit like the Holy Grail: it seems enticing and everyone wants it and thinks it's a magic solution, but its usage turns out to be incredibly limited. It is only useful within a cave of specific situations, and it maybe wasn't worth all the effort and stress and agony that Indiana Jones and his dad put themselves through to find it, even though it was good to see John Rhys-Davies back again.

676 (edited by pneumatic 2023-03-27 18:04:48)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

ireactions wrote:

I'm getting some moire patterns where there were previously jagged edges. This screenshot from "Eggheads":

https://i.ibb.co/Kq01GDp/Sliders-107-Eggheads-mp4-snapshot-26-09-757.png

Any recommendations from pneumatic for what addresses this? Or is it best left alone?

It's a side effect of field interpolation, which is what deinterlacing does, and QTGMC is a deinterlacer that suffers from this as well.   I had mentioned it here (check out the sleeve on her shirt).

pneumatic wrote:

By the way QTGMC is effective at suppressing the field alignment issue too, and looks sharper [1][2][3] but introduces moire artefacts on stippled patterns [4].

The only way I could suppress it is to blur/blend pixels with the one above/below (which is the left side of that 4th image comparison) but then the image becomes quite a lot softer to the point that TIVTC alone would be the clear winner imo.

edit: just checked that scene with TIVTC and it's no better, cause the wall texture has alternating 1px patterns which gets falsely detected as combing, triggering TIVTC to switch to deinterlacing mode for those frames, resulting in moire.    Setting the combing detection threshold higher isn't an option as I've already set it so high that there are some false negatives creeping in.   The way this would be handled by a restoration studio would be to flag that scene as progressive to force deinterlacing to not kick in just for that scene.   A better solution would be to have better more reliable combing detection in the first place.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

Thanks. I'll stick with the moire patterns. It's fine. I'm redoing all the encodes again with Lanczos at four taps.

I don't know why, but I don't like the results with the Nvidia encoder. GPU acceleration is fast (7 - 10 minutes per episode), but image just looks dull and fuzzy to me, even though when I take screenshots, I can't see any difference. I'm sticking to CPU encodes (an hour per episode) for this.

I may need to design some artwork for SLIDERS: SEASON 1: THE PNEUMATIC EDITION.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

ireactions wrote:

Thanks. I'll stick with the moire patterns. It's fine.

For that particular episode QTGMC would be better imo as it preserves all the 60i sections where they're playing Mindgame - those should look buttery smooth 60fps.

QTGMC has a ton of spatial smoothing settings, I'll try having a play around with them to reduce moire, but I've got a feeling it would come at the expense of sharpness.   

ireactions wrote:

I don't know why, but I don't like the results with the Nvidia encoder. GPU acceleration is fast (7 - 10 minutes per episode), but image just looks dull and fuzzy to me, even though when I take screenshots, I can't see any difference. I'm sticking to CPU encodes (an hour per episode) for this.

Is the screenshot on a static scene or motion?

679 (edited by pneumatic 2023-03-28 15:55:36)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

New discovery...QTGMC has a separate "repair progressive" mode which can provide a significant improvement when applied to all frames post-TIVTC.  This mode doesn't deinterlace at all, it just "repairs" progressive frames in a way that reduces deinterlacing artefacts, aliasing, moire, shimmer etc. without smudging out any resolution in the source. I ran it through a resolution test pattern and 1px alternating white and black lines retain full contrast.

The downside is that it doesn't provide any benefit to video that was already deinterlaced by QTGMC, but it greatly reduces post-TIVTC deinterlacing artefacts such as:

1. "Eggheads" the moire on the wall shingles are suppressed - clip
2. "Prince of Wails" the Jeep radiator grille is now perfectly rendered
3. "Last Days" opening street scene with the tree branches is perfectly rendered

Because it only works post-TIVTC, you have to forfeit the perfect 60fps frame pacing that you would get with QTGMC deinterlacing only, such as the scenes in "Eggheads" where they're playing Mindgame (comparison: QTGMC 1080p60 , TIVTC + QTGMC repair 1080p24)

Anyway, I'm super stoked with this discovery as it means I don't have to worry so much about false positives with TIVTC's combing detection, as those frames will come out looking nice after being repaired by QTGMC.   I'm also using it on Rugrats (streaming service version) and it beautifully smooths out all the deinterlacing artefacts baked into it.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

May have found a good balance with an alternative QTGMC solution.  It's like the original QTGMC solution but resolves a bit more resolution when camera is still.

Pros:
* full 480p resolution on static elements (credits text, moments when the camera is perfectly still etc.)
* QTGMC smoothing and antialasing on moving parts of the image (also reduces moire more than QTGMC alone)
* perfect frame pacing (spinning earths and Mindgame sections are 30fps & 60fps)

Cons:
* no smoothing of the field alignment issue on static elements (since those are just weaved to 480p)

Preview: bwdif deinterlace + qtgmc repair.mkv

681 (edited by RussianCabbie_Lotteryfan 2023-03-28 15:52:33)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

pneumatic wrote:

May have found a good balance with an alternative QTGMC solution.  It's like the original QTGMC solution but resolves a bit more resolution when camera is still.

Pros:
* full 480p resolution on static elements (credits text, moments when the camera is perfectly still etc.)
* QTGMC smoothing and antialasing on moving parts of the image (also reduces moire more than QTGMC alone)
* perfect frame pacing (spinning earths and Mindgame sections are 30fps & 60fps)

Cons:
* no smoothing of the field alignment issue on static elements (since those are just weaved to 480p)

Preview: bwdif deinterlace + qtgmc repair.mkv

Looks better overall to me.   Compared the two files (qtgmc.mkv to this one)

One question, when you compare the reds between the two videos, are they the same for you?

edit: here's a before/after:  https://imgsli.com/MTY1NDY1

For  reference, look at Arturo's skin pigment and red coat.

682 (edited by pneumatic 2023-03-28 16:15:16)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

RussianCabbie_Lotteryfan wrote:

One question, when you compare the reds between the two videos, are they the same for you?

edit: here's a before/after:  https://imgsli.com/MTY1NDY1

For  reference, look at Arturo's skin pigment and red coat.

Yep, it's way off on the old SD one, and I'm pretty sure it's cause your media player isn't converting SD colours to HD colours to suit your display mode which uses HD colours.

But that's my fault cause I assumed that media players would see the file's resolution is SD and convert them to HD colours, as that's how mine operates - https://imgsli.com/MTY1NDcz

I'll run off another copy with the mkv file properly tagged for SD colours and then it should look fine on your end and all media players.  Sorry about that - good catch.

edit: btw, the colour is still slightly different on mine too, and I think it's cause my media player uses slightly different formulas to do the conversion compared to the Avisynth filter I used.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

pneumatic wrote:
RussianCabbie_Lotteryfan wrote:

One question, when you compare the reds between the two videos, are they the same for you?

edit: here's a before/after:  https://imgsli.com/MTY1NDY1

For  reference, look at Arturo's skin pigment and red coat.

Yep, it's way off on the old SD one, and I'm pretty sure it's cause your media player isn't converting SD colours to HD colours to suit your display mode which uses HD colours.

But that's my fault cause I assumed that media players would see the file's resolution is SD and convert them to HD colours, as that's how mine operates - https://imgsli.com/MTY1NDcz

I'll run off another copy with the mkv file properly tagged for SD colours and then it should look fine on your end and all media players.  Sorry about that - good catch.

edit: btw, the colour is still slightly different on mine too, and I think it's cause my media player uses slightly different formulas to do the conversion compared to the Avisynth filter I used.

I'm using VLC, which is obviously a pretty sophisticated media player..

In the old SD one, are you suggesting that it isn't doing shades of colors well, so it sort of averages out a given tone to the nearest SD color (which may be more pronounced)?

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

RussianCabbie_Lotteryfan wrote:

In the old SD one, are you suggesting that it isn't doing shades of colors well, so it sort of averages out a given tone to the nearest SD color (which may be more pronounced)?

There are 2 issues:

1. SD and HD use different formulas to convert between RGB and YCbCr.  Video is encoded in YCbCr and the media player will convert it to RGB internally so it has to use the right formula.

2. HD and SD use different gamuts (range of colour) which basically defines what wavelengths the RGB values correspond to. 

On top of this there is also some discrepancy in how different media players and renderers implement the conversions, and what assumptions they make about the monitor's properties in the process.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

Here's that first qtgmc preview again, but this time tagged with SD colours - specifically "-colorspace 6 -color_primaries 6" in ffmpeg (6 = rec.601/smpte170m)

qtgmc 720x540p60 Rec.601 tagged.mkv

I'd expect VLC should read the tags and do the conversion properly.  Without the tags VLC probably just assumed nothing and didn't do any conversion, which is fair enough.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

pneumatic wrote:

Here's that first qtgmc preview again, but this time tagged with SD colours - specifically "-colorspace 6 -color_primaries 6" in ffmpeg (6 = rec.601/smpte170m)

qtgmc 720x540p60 Rec.601 tagged.mkv

I'd expect VLC should read the tags and do the conversion properly.  Without the tags VLC probably just assumed nothing and didn't do any conversion, which is fair enough.

yes, now the colors of qtgmc.mkv match the bwdif + qtgmc repair.mkv

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

Or was the non hd color version more how it played on the old tube tvs?

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

Would pneumatic be able to share the script for TIVTC + QGTMC repair? I can't seem to get QGTMC working on Hybrid at all.

Are you running the MKV through TIVTC and then encoding the resulting MP4 file again with QGTMC? Or is it all one job with the MKV to MP4?

I don't expect to have time to learn how to use Avisynth+ until the weekend. It would be nice for The Pneumatic Edition on my end to not have the moire patterns.

Thank you again for sharing all your expertise and ongoing learnings with us.

689 (edited by pneumatic 2023-03-29 12:15:23)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

ireactions wrote:

Would pneumatic be able to share the script for TIVTC + QGTMC repair? I can't seem to get QGTMC working on Hybrid at all.

clip = "C:\S01E08 Eggheads.mkv"   # produced from the disc using MakeMKV (lossless remux)
CPUcores = 4  

video = LWLibavVideoSource(clip, stream_index=-1, repeat=true, cache=true)
audio = LWLibavAudioSource(clip, stream_index=1, cache=true)
AudioDub(video, audio)

TFM(mode=0, slow=2, scthresh=100, PP=3, metric=1, cthresh=9, MI=128,
\    clip2=PropDelete("_FieldBased").bwdif(field=-1, thr=5, edeint=nnedi3(field=-1)))

TDecimate(mode=0, cycleR=1, cycle=5, hybrid=1, viddetect=2, vidthresh=4.5,
\    denoise=true, chroma=false, hint=false)

QTGMC(InputType=1, preset="slow", Sharpness=0.0, FPSDivisor=1, EdiThreads=CPUcores/2) 

z_ConvertFormat(                      
\ colorspace_op="601:601:170m:full=>709:709:709:full",
\ resample_filter="spline36",
\ interlaced=true,
\ dither_type="ordered")

LanczosResize(1440, 1080)

# Optional: the following Trim command will output only a short snippet, useful for quick previewing:
# Trim(0, 240) # outputs the first 240 frames only (10 seconds @ 24fps)
# Trim(240, 480) # outputs frames 240-480 
# Trim(480, 0) # outputs frames 480 til end
# ShowFrameNumber() # overlay frame number onto video

Prefetch(CPUcores)

edit: alternatively, if you want the same moire reduction as above, but with original 60fps frame pacing (smooth on spinning earths, Mindgame sections etc.) just delete both the TFM and TDecimate lines and put instead:

propDelete("_FieldBased").bwdif(field=-2, thr=5, edeint=nnedi3(field=-2))

The downside is that all frames will be deinterlaced, so you'll only get true 480p resolution on static elements but not on moving elements.


ireactions wrote:

Are you running the MKV through TIVTC and then encoding the resulting MP4 file again with QGTMC? Or is it all one job with the MKV to MP4?

All in the one job, otherwise image quality won't be quite as good due to multiple rounds of compression.  I'm just opening the windows command prompt, changing to the ffmpeg folder with "cd c:\program files\ffmpeg\bin", then pasting this and hitting enter.  ffmpeg takes the Avisynth script as the input and produces the mkv file as the output.   

ffmpeg -color_range 1 -colorspace 1 -color_primaries 1 -i "C:\TheAboveScript.avs" -c:v libx264 -tune film -b:v 8000000 -pix_fmt yuv420p -color_range 1 -colorspace 1 -color_primaries 1 "TIVTC + QGTMC repair.mkv"


ireactions wrote:

Thank you again for sharing all your expertise and ongoing learnings with us.


Happy to help, and thanks for everyone's help as well.



RussianCabbie_Lotteryfan wrote:

was the non hd color version more how it played on the old tube tvs?

Well the show would have been produced with rec.601/smpte-170m colourimetry, so you'd want VLC to be interpreting it as such, and that seems to be the case with the tagged mkv.  All other mkv's I uploaded to gdrive after that were correctly tagged as well. 

I'm not familiar with VLC but there should be some extra status info or debug screen to get VLC to display details about the colourimetry it's using.  Or you could take a full screenshot on the scene with the front view of the red car and compare it to my screenshot which is using correct colourimetry as indicated by the overlay on my screenshot.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

Actually, I found that turning off the post processing in TIVTC eliminated the moire pattern:

https://i.ibb.co/GVssgXB/Sliders-107-Eggheads-mp4-snapshot-26-10-216.png
Settings:

https://i.ibb.co/4dM9bvH/hybrid-1.jpg
https://i.ibb.co/5rkY68v/hybrid-2.jpg

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

That's true, but then you will have some combed frames on certain sequences like during the intro, or certain scene changes that weren't spliced correctly, or when they're playing Mindgame in Eggheads.   

If occasional combing doesn't bother you then you should use it as it retains full 480p resolution on 24fps and 30fps sequences which make up 98% of the show. 

If TIVTC had more reliable combing detection this wouldn't be an issue.  We can probably improve it for this episode by playing with the combing detection thresholds (cthresh and MI parameters) but in my experience this only tends to provide a choice between false positives or false negatives.

But it's still an improvement from no post processing at all, so try setting MI=200 which eliminates false positives from that street scene (no moire).  The highest number of combed pixels for that scene was 195 inside the square by the window, so setting MI to 196 or above would avoid moire on that scene:

https://i1.lensdump.com/i/TNY1XM.png

Just be aware you may get other scenes that could be combed and have less than 200 combed pixels and those would go undetected (false negative) and remain visibly combed.

To get that debug overlay in Avisynth: TFM(PP=1, display=true)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

Hot damn! More experiments!

I'm looking at the Hybrid + Avisynth preview and at MI:200 with post processing re-enabled, I'm still seeing a few combed frames in the opening titles, but definitely not as many as without post-processing. I've loaded all the MKVs back into the queue. Atomic batteries to power. Transcoders to speed.

693 (edited by pneumatic 2023-03-30 21:47:04)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

ireactions wrote:

I'm looking at the Hybrid + Avisynth preview and at MI:200 with post processing re-enabled, I'm still seeing a few combed frames in the opening titles, but definitely not as many as without post-processing.

Turning off PP turns off combing detection, so you should be getting more combed frames not less.

Increasing MI means there has to be more combed pixels detected within a frame before it declares the frame as combed, so expect to see more combed frames when raising it.  The default is 80 which produces a lot of false positives (like the wall texture it thinks is combing) so by default the error is biased in favour of deinterlacing artefacts (eg. moire) instead of combing artefacts.   In my experiments the highest I could go without too many false negatives was 128, which is half the pixels in a 16x16 block.

Hard for me to say what's going on...maybe Hybrid has implemented it different or wrongly?  The combing threshold option in Hybrid appears is mislabeled as "chroma threshold", maybe there are other errors in the implementation.

694 (edited by ireactions 2023-03-30 22:54:54)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

pneumatic wrote:

Turning off PP turns off combing detection, so you should be getting more combed frames not less.

I'm confused. I turned PP back on, so wouldn't I be getting fewer combed frames?

I can confirm that with PP enabled and MI set to 200, most frames that were previously combed are now clear:

https://i.ibb.co/kmVvphg/compare.jpg

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

Sorry, looks like I had a brain fart and misread what you wrote roll

696 (edited by ireactions 2023-03-31 06:52:41)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

Please be more careful, pneumatic. It has become readily apparent that you are the only person in this thread who knows what he's doing. The only person here who is able to identify a goal, devise a plan and execute each stage to accomplish your ends.

When you slip, it heralds a dark era where nobody here can get anything done because pretty much everyone here is just pressing random buttons and checking or unchecking boxes haphazardly with no understanding of the underlying technologies and processes and datastreams at work and vaguely hoping that something will transcode well or even at all.

You are literally the only one in this thread who has his crap together. You must never, ever, ever make a mistake again.

(This was a joke.)

On a more serious note, do you have any recommendations/suggested filters/advised settings for adding film grain? A little grain texture would be nice to offset the soft-focus look.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

Na you're both artists and craftsmen and preservationists and poets!

It's thrilling to have a corner of humanity that takes on these interests and values.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

ireactions wrote:

On a more serious note, do you have any recommendations/suggested filters/advised settings for adding film grain? A little grain texture would be nice to offset the soft-focus look.

Funnily enough QTGMC has some dedicated settings for "restoring noise": http://avisynth.nl/index.php/QTGMC#Nois … _Denoising

There's also some "AddGrain" filters here: http://avisynth.nl/index.php/External_filters#Effects

I haven't played with any of them yet - currently spending the day chasing data corruption on one of my drives containing all my remuxed DVDs, including Sliders.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

Question for Ireactions and pneumatic: What are your favorite settings on Topaz? I'm planning on getting into the Sliders episode 4K conversion game this weekend. Using my Mac Studio M1 as the video conversion machine, along with Topaz AI to work on converting seasons 1-4.

700 (edited by RussianCabbie_Lotteryfan 2023-04-01 11:25:18)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

@quinnslider  it seems like artemis, artemis dehalo (is that the same thing?) and gaia may be the top algorithms.... ib used a lot of artemsis bouncing between different quality settings depending on the season, i believe.  his process is written up on page 1 of this thread....

(also gaia is slower but you may have enough processing power to use it -- it may be more ideal for later seasons).


On a general note, picking up on some of what ib was saying on limitations with topaz, I was thinking the next great future technology might not focus on existing film grain, but rather re-build the image altogether.  taking the input of what you have given it, comparing to what a house in HD should look like, or a grey coat, or a piece of furnature, and rebuilding each element in the frame, trying to closely match its form and lighting but also recognizing what in HD it should look like.    as ib has explained, currently a solution like topaz is grabbing on the grain in the image to understand it better and building out from there, but when you have sources that lack grain (which Sliders s1 is limited on outside of the pilot) it's hard for topaz to get much more out of the image in terms of filling in missing lines  of resolution.    i will say it was much surprior to what an avisynth script was able to do with my brother's old documentary footage, so I think it is indeed quite an accomplishment.   but in the future of any of the technology for this use case may be more similar to the AI image generators we have now, only 1000x better and better able to accurately reconstruct what a 4k scene should reasonably look like based on a low quality source.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

QuinnSlidr wrote:

Question for Ireactions and pneumatic: What are your favorite settings on Topaz? I'm planning on getting into the Sliders episode 4K conversion game this weekend. Using my Mac Studio M1 as the video conversion machine, along with Topaz AI to work on converting seasons 1-4.

I would suggest using the Artemis setting on Low Quality, Noise setting for Seasons 2, 4 and 5 and Artemis on Medium for the Pilot and Season 3. Also, it's necessary to add AI grain to the upscale to avoid a waxy quality to the upscaled image (set it to 3 - 4 on Amount and 1 - 2 on size depending on your preference by doing previews of wide shots, medium shots and closeups). I do not find Topaz AI to be effective for Episodes 1.02 - 1.09, unfortunately.

As I said before: Topaz AI is like the Holy Grail where everyone thinks of it as a magical cure-all,  but it's actually only useful in specific situations of isolated parameters and Indiana Jones and his dad probably went home wondering if it was really worth the trouble of finding it.

The issue with Topaz: it is only effective for making high quality SD video look good in HD. It can't make a bad SD video look better in HD. It can't even make a bad SD video look better in SD.

AI upscaling operates by scaling image detail with algorithms designed to recognize a wide variety of textures and add additional pixels in a manner that's suited to blend in smoothly with each texture. Blockiness gets smoothed out, sharp details remain intact at the higher resolution. AI upscaling also analyzes the film grain content and uses that to rebuild an approximation of the original film image. That grain can get refined into an overly smooth texture, so it's necessary to add grain back on to avoid the watercolour look.

Modern TVs and video player software can generally stretch SD video to HD screens well, but what was sharp and detailed in SD becomes a little fuzzier and less resolved in HD. AI helps stretched SD video maintain its original strengths, but AI can't add merits the video didn't already have.

When it comes to upscaling SLIDERS, it depends on which box sets you have. I personally think upscaling to 4K would be overstretching these old digital videotape files and have aimed for the 720p - 1080p range instead.

The Pilot was shot and edited on film, so it has a crisp level of detail and grain content in the Universal disc that benefits from AI upscaling to 1080p. This is not the case with the Mill Creek and Turbine sets. Mill Creek has overcompressed the files and there is not enough SD grain or detail for Topaz. Turbine uses the PAL masters and since those were drawn on analog videotapes, they are stretched to the higher PAL resolution and even blurrier than the NTSC version.

Episodes 1.02 - 1.09, while shot on film, were unfortunately transferred to analog videotape for editing. This low resolution format has no grain and little detail, so Topaz can't rebuild the images. It always looks like watercolour. These episodes look bad on the Universal discs, very bad on the Turbine discs and terrible on the Mill Creek discs.

Season 2, thankfully, made the switch to early digital videotape which preserves detail and film grain in a downscaled form. There's also a slightly blocky texture to the digital videotape format of the era. The Universal and Mill Creek discs of Season 2 are good enough to upscale to 720p, but results will be sharper with Universal. The Turbine version uses the higher resolution PAL masters with little compression, and those would allow 1080p output. Season 2 will be markedly improved from non-AI scaling because AI can smooth out the digital videotape texture while rendering the grain into detail.

Season 3 used a newer form of digital videotape that didn't preserve as much of the film grain but is well-detailed enough on the Universal and Mill Creek discs to upscale to 720p. The Universal-based upscale will be preferable. The Turbine version of Season 3, however, is significantly sharper and would look good at 1080p. However, because Season 3 is already quite sharp, it doesn't really benefit much from AI upscaling and the AI upscales I've done don't look significantly different from a Lanczos or even a bicubic scaling conversion.

Seasons 4 - 5 were shot on 16mm film with an extremely high grain level that's very present even in a downscaled, compressed SD image. Because of this, Topaz AI can rebuild the original film image. The Mill Creek version can be upscaled to 720p extremely well. The Universal and Turbine files can all be upscaled to 1080p. The Universal upscale will look very good and the Turbine upscale will look even sharper. It will look different from the SD files, less grainy and more resolved and detailed.

I haven't been happy with any AI upscales of Season 1's post-Pilot episodes and would just stick to pneumatic's methods (TIVTC and QTGMC deinterlacing-decombing with Lanczos scaling it to 1080p). It will look like soft-focus DVD, but that's the best that the post-Pilot Season 1 files allow.

After experiencing all of the above, I have really lost a lot of enthusiasm for AI upscaling SLIDERS. Seasons 2 - 5 on the Turbine set look fine with scaling to fill a 55 inch TV. At living room distance, the artifacts don't really matter.

I don't want to do any more upscaling with current methods because I don't want to produce a near-HD image of "The Breeder" and its like while episodes like "Eggheads", even after pneumatic's work, still only look like an 'okay' DVD file.

Considering the original state of those Season 1 files (blurriness, comb lines, jagged edges), it's a miracle that pneumatic has achieved watchable results. And until we have a way to make look "Eggheads" look like a decent 720p release, I don't really have any further interest in upscaling the rest of the show. It is a crime against culture that "Slither" can look 1080p presentable while "The Weaker Sex" looks like VHS.

702 (edited by pneumatic 2023-04-02 15:52:09)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

ireactions wrote:

Considering the original state of those Season 1 files (blurriness, comb lines, jagged edges)

Just to clarify regarding comb lines or "combing", those aren't defects - interlaced video is supposed to have combed frames.  It doesn't necessarily have to contain combed frames, but it's expected and normal if it does have combed frames.

The way I'd summarise it is like this: 480i30 is the same as 480p30 in that it still has 30 frames per second of 720x480, except with 480i30 each frame may or may not contain two different images inside it - one in the even lines and the other in the odd lines (top field and bottom field respectively).  If such frames are displayed "as is" then it looks combed as we're seeing two different images from two different points in time weaved together. CRT's would display each field seperately one after the other so the resulting image wouldn't look combed.

Not all frames in the source are necessarily combed though, eg. the spinning earths intro animation is effectively 480p30 - both even and odd lines are from the same image, so it doesn't even need any processing at all and will look identical to 480p30.   The majority of the show uses a pattern of 3 progressive frames followed by 2 combed frames - a lossless method of storing 24p inside a 30i container. 

In an ideal world the entire show would just follow this predictable 3-2 pattern (cadence), but in practice the source may contain a hodge podge of different cadences and/or random cadence breaks. eg. some sequences where the camera is panning with the 3-2 pattern, and then they'll overlay credits half way through a frame, momentarily breaking the 3-2 pattern. Some frames either side of a scene change may have unexpected combed frames due to splicing half way through a frame.  In Eggheads when they're playing Mindgame, all frames are combed. 

Dealing with all of these cadence changes presents a massive headache if your goal is to figure out which fields can be weaved with others to extract as many progressive frames as possible.  TIVTC does this quite well, but inevitably ends up with leftover combed frames which contain fields that couldn't be weaved with any others due to the aforementioned idosyncrasies.  This is where TIVTC's "post processing" comes in - it detects and deinterlaces any such combed frames leftover after the field matching process.  This relies on combed frame detection which is prone to false positives or false negatives.   But when a combed frame is detected, it uses motion compensated deinterlacing which maximises resolution by weaving parts of the image which are static, and upscaling ("interpolating") parts which are moving.  So anything that moves basically drops to 240p, but anything static has full 480p resolution.  The drop to 240p on motion causes aliasing, shimmer and moire, but QTGMC's "repair progressive" mode (InputType=1) does a remarkable job at cleaning this up and making it look like 480p (except for moire patterns). 

Separate to this is QTGMC's deinterlacing mode (InputType=0) which is intended to be used on its own and not in conjunction with TIVTC. In this mode, moire is present even on static elements, because QTGMC deinterlacing never weaves fields to progressive frames - it is a completely novel solution which upscales every field from 240p to 480p, and then does some very clever blending with neighbouring fields, using motion vectors and neural network antialiasing on edges to smooth everything out.  The actual amount of resolved resolution on static elements is somewhere around 360p according to a test pattern.  On moving objects it visually appears similar to 360p as well, although I haven't measured it.  This is great for low quality sources which never contained 480p resolution to begin with (especially the Mill Creek DVDs as it blends and smooths the field alignment issue) but for higher quality sources I prefer its "repair progressive" mode (InputType=1) applied to the motion compensated deinterlacing.

For Season 1 I will probably use TIVTC + post processing motion compensated deinterlacing + QTGMC "repair progressive" mode.  The repair progressive does slightly suppress film grain though - on higher quality sources it does tend to look a little "processed".  But it does get rid of aliasing and shimmer on those occasional deinterlaced frames to the point where I have a hard time telling whether I'm looking at TIVTC'd frames or deinterlaced frames, except for high frequency patterns which still have noticeable moire.

703 (edited by ireactions 2023-04-02 16:41:51)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

I wanted to try the QTGMC repair processing, but... I can't seem to get it together.

I installed AviSynth+, dropped in the DLL files as advised, and then ran pneumatic's script in AviSynth, but I got a message saying: "Script error: There is no function named 'QTGMC'." I'm finding the instructions on the Wiki to be just... baffling. http://avisynth.nl/index.php/QTGMC

Once again, I feel that pneumatic is the only one here who knows what he's doing.

Anyway. I'm trying a new Hybrid-baased, Vapoursynth-powered transcode of 1.02 - 1.09 with pneumatic's TIVTC and QTGMC recommendations and adding some additional saturation and grain to fill in some of the soft-focus lack of detail.

**

Something I've found: the Turbine blu-ray set, using the PAL masters, has extremely blurry versions of the Season 1 episodes. All of Season 1, including the Pilot, are noticeably blurrier than the Universal set and a little better than the Mill Creek set. However, the Turbine PAL files do not have the jagged edges and comb lines and of the Universal and Mill Creek discs. Did pneumatic find this to be the case in the Via Vision PAL DVDs?

Turbine's team can't remember what they did to process the PAL files, but I wonder if they detelecined the PAL files in a manner that created image quality degradation or if the PAL files from Universal were produced when the technology had improved and didn't have the fields scaled independently from the videotapes when converting them to DVD.

704 (edited by pneumatic 2023-04-09 22:48:29)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

ireactions wrote:

I wanted to try the QTGMC repair processing, but... I can't seem to get it together.

I installed AviSynth+, dropped in the DLL files as advised, and then ran pneumatic's script in AviSynth, but I got a message saying: "Script error: There is no function named 'QTGMC'." I'm finding the instructions on the Wiki to be just... baffling. http://avisynth.nl/index.php/QTGMC

Yeah sorry about that - the dlls I linked to included everything except QTGMC.   

QTGMC has a ton of dependencies, it needs about a dozen other dlls which can be found under the "requirements" section of the above link.   For convenience I've put everything in a zip file here.   

The one that goes in C:\windows\system32 is from this plugin used by QTGMC (so FYI so that you know it's legitimate).

The rest can all be dropped in eg. C:\Program Files (x86)\AviSynth+\plugins64 and then it should work.

If you have a media player that supports opening avisynth scripts (eg. MPC-HC)  then you can just open your avisynth script directly in the media player and begin previewing them straight away.  This is how I'm using it and saves a ton of time, would highly recommend.

For example make a little test avisynth script, right click on the desktop and "create new avisynth script", open it in a text editor and type this in it:

Version()

Then double click the avisynth script and it should open a video in the media player like this:

https://i.lensdump.com/i/Tf0CtD.png

Basically "Version()" created a 10 second video clip containing that text overlay.

If you don't have MPC-HC I'd recommend this pack which contains it along with numerous codecs and the acclaimed MadVR video renderer: https://codecguide.com/download_k-lite_ … k_full.htm

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

Thanks. MPC-HC was the only part I got right. The Kazaa Codec Pack has always been my go-to. VLC is just too odd in how it positions itself at bizarre sizes every time I open a video file with it. MPC-HC has the sense to put itself in the center of the screen and the decency to use hardware acceleration.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

RussianCabbie_Lotteryfan wrote:

@quinnslider  it seems like artemis, artemis dehalo (is that the same thing?) and gaia may be the top algorithms.... ib used a lot of artemsis bouncing between different quality settings depending on the season, i believe.  his process is written up on page 1 of this thread....

(also gaia is slower but you may have enough processing power to use it -- it may be more ideal for later seasons).


On a general note, picking up on some of what ib was saying on limitations with topaz, I was thinking the next great future technology might not focus on existing film grain, but rather re-build the image altogether.  taking the input of what you have given it, comparing to what a house in HD should look like, or a grey coat, or a piece of furnature, and rebuilding each element in the frame, trying to closely match its form and lighting but also recognizing what in HD it should look like.    as ib has explained, currently a solution like topaz is grabbing on the grain in the image to understand it better and building out from there, but when you have sources that lack grain (which Sliders s1 is limited on outside of the pilot) it's hard for topaz to get much more out of the image in terms of filling in missing lines  of resolution.    i will say it was much surprior to what an avisynth script was able to do with my brother's old documentary footage, so I think it is indeed quite an accomplishment.   but in the future of any of the technology for this use case may be more similar to the AI image generators we have now, only 1000x better and better able to accurately reconstruct what a 4k scene should reasonably look like based on a low quality source.

Thank you RCLF, pneumatic, and ireactions.

I've been experimenting with the Artemis filter over the weekend along with the above settings for upconverting to 4K and the results are fantastic on the Pilot.

However, I have not been able to find the 6 hours required to stick with my computer to process this, so I will probably be doing a full process this upcoming weekend.

Thank you all for your feedback!!

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

When running upscale jobs, they generally had to happen while I slept. I have 32GB of RAM on my gaming laptop, so that allowed me to do my other work even while Topaz monopolized the GPU.

Currently, I have Hybrid running a transcode job on Episodes 1.02 - 1.09 with pneumatic's advised settings, nnedi3 scaling to 1080 and grain to fill in some of the fuzziness. It's taking about 3 - 4 hours per episode. I just had a look at "Prince of Wails" and "Fever" which finished, putting them on a very mediocre 55 inch LED TV.

It occurs to me that in two episodes in a row, Wade has to nearly get hit by a car to get the plot going.

They look okay to me, and as I've considered the Universal files to be awful, okay is quite an accomplishment. Most of the distortions from viewing these episodes on a modern TV (lines, jagged edges) are gone. The soft focus look is a little more deblurred now. Mediums and closeups look good. Really wide shots still betray that this is more a videotape than a digital file, but it's still okay.

The odd distortions of Topaz AI upscaling are absent. Yes, Topaz AI's closeups looked better, but the mediums and wides looked so bad that it was distractingly inconsistent. The Pneumatic Edition is consistent. The grain and the nnedi3 scaling are making the lack of detail tolerable, and pneumatic's instructions have reduced the jaggies and comb lines to the point where their occasional appearances seem acceptable. I've upped the colour a bit just to make it look a less faded.

I could live with these.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

pneumatic wrote:

Well, I'm officially stumped.   I'm looking at the opening scene of S01E06 Last Days from the Universal NTSC DVD (disc ISO, not someone else's transcode) and I'm seeing deinterlacing artefacts BAKED INTO the source.  These artefacts are not present in another copy of this episode which was transcoded by someone else....
I'm not sure what's going on, either there are multiple Universal NTSCs with different image quality, or whoever did this particular transcode of it is wielding some voodoo magic video processing that I cannot get my head around.

For completeness here's a short clip of that scene with vs without QTGMC repair - the difference is quite astonishing: https://drive.google.com/file/d/1D4PEbu … share_link

709 (edited by RussianCabbie_Lotteryfan 2023-04-05 12:46:36)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

pneumatic wrote:
pneumatic wrote:

Well, I'm officially stumped.   I'm looking at the opening scene of S01E06 Last Days from the Universal NTSC DVD (disc ISO, not someone else's transcode) and I'm seeing deinterlacing artefacts BAKED INTO the source.  These artefacts are not present in another copy of this episode which was transcoded by someone else....
I'm not sure what's going on, either there are multiple Universal NTSCs with different image quality, or whoever did this particular transcode of it is wielding some voodoo magic video processing that I cannot get my head around.

For completeness here's a short clip of that scene with vs without QTGMC repair - the difference is quite astonishing: https://drive.google.com/file/d/1D4PEbu … share_link

wow.  this is what i kept saying about old sliders content looking like it was poorly kept film stock or 3rd generation vhs footage.  the undulating waterfall effect, the vibrating lines.  the crappy, faded color.

that's definitely a lot cleaner & more stable.

710 (edited by pneumatic 2023-04-05 17:47:35)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

Yep, and the good thing is it doesn't smudge out resolution.   It does suppress film grain and video noise to a slight degree though, but 1px thick lines remain fully resolved, unlike QTGMC's deinterlacing mode.  Film grain can be added back in with "EZKeepGrain=n"  (n: 0.0 = off, 1.0 = a good starting point). 

Also when using QTGMC in any mode I would recommend adding the parameter "Rep0=13" which repairs motion vector errors for a very minimal CPU expense.  It seems to only be an issue when QTGMC is processing duplicate frames where the motion vectors seem to "bleed through" into the next duplicate frame when there is motion:  default , Rep0=13  .

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

Do Rep0=13 and EZKeepGrain require additional DLLs?

I put your DLL pack into the Avisynth folder, but it still didn't run until I added more DLLs (avsresize, BWDIF, LSMASHSource, nnedi3, TDeint, TIVTC). I can't seem to get Rep0=13 and EZKeepGrain running after adding them to your script.

I'm hoping to spend some of the Easter holiday reading up on Avisynth+ and how to use it so that I'm not dependent on the only person in this thread who knows what he's doing to type out the code.

I'm current experimenting with different Hybrid scaling solutions and grain settings until I'm happy.

Hybrid is doing a good job with most episodes except "Last Days" which remains very jagged and ugly in the opening scenes, so that's probably an Avisynth+ job.

712 (edited by pneumatic 2023-04-09 04:42:28)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

ireactions wrote:

Do Rep0=13 and EZKeepGrain require additional DLLs?

I don't believe so, for example QTGMC(preset="slow", Rep0=13, EZKeepGrain=1.0) is working on my system with just those dll's I put on gdrive.  I double checked by removing all other dll's from my C:\Program Files (x86)\AviSynth+\plugins64 folder and putting only those ones on gdrive in there. 

What error message do you get when doing eg. QTGMC(preset="slow", Rep0=13, EZKeepGrain=1.0)?

I do believe QTGMC may have some extra dependencies when using "very slow" and "placebo" presets though - those should be available in "optional plugins" here: http://avisynth.nl/index.php/QTGMC#Requirements


ireactions wrote:

I put your DLL pack into the Avisynth folder, but it still didn't run until I added more DLLs (avsresize, BWDIF, LSMASHSource, nnedi3, TDeint, TIVTC).

Yes, so you'll need:

1. TIVTC dll's: https://drive.google.com/file/d/1IgIze3 … FOW5F/view
2. QTGMC dll's: https://drive.google.com/file/d/1c1Duyt … YnXZx/view

But I forgot about avsresize.dll - that one is needed to do the colourimetry conversion from SD colours to HD colours if you are upscaling in Avisynth.

ireactions wrote:

I'm hoping to spend some of the Easter holiday reading up on Avisynth+ and how to use it so that I'm not dependent on the only person in this thread who knows what he's doing to type out the code.

The first thing I'd do is confirm you've got Avisynth up and running by making that hello world Version() clip.

Once you've got that running in MPC-HC, the next thing I'd recommend is set MPC-HC's audio render to "MPC Audio Renderer" as that seems to be required for compatibility with Avisynth audio (in MPC-HC options -> playback -> output -> audio renderer -> MPC Audio Renderer).

Also in MPC-HC options -> internal filters -> source filters -> tick Avisynth.  This will make MPC-HC use its own internal LAV filter to open .avs files, which in my experience is better than the default Microsoft one you will probably get if it's unticked.  If you have any troubles, right click the video image while an .avs file is playing -> Filters -> copy filters to clipboard, and paste the result here.

If you can get to this stage then you should be good to go with writing Avisynth scripts and previewing them in realtime without having to spend hours transcoding to mp4.   QTGMC's "medium" or "fast" presets may be useful too.  Don't forget to enable multithreading with Prefetch(cores) at the end of the script, as QTGMC needs multithreading to be fast enough for realtime use.

I'm currently working on a script that will default to outputting 60fps deinterlaced frames thus preserving frame pacing.  Then if a deinterlaced frame is similar to the TIVTC'd frame within a threshold, swap it for the TIVTC'd frame.  This should give the best of both worlds - TIVTC 480p resolution on film sections (most of the show) and autoswitching to 60fps deinterlaced goodness on video sequences (like Eggheads mindgame, intro sequence, etc.).

edit: here's a preview.  Still a work in progress, trying to adjust the thresholds to handle the switching behaviour on low motion

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

Your original code QTGMC script works for me:

QTGMC(InputType=1, preset="slow", Sharpness=0.0, FPSDivisor=1, EdiThreads=CPUcores/2)

This, one, however:

QTGMC(preset="slow", Rep0=13, EZKeepGrain=1.0)

Gives me this error:
https://i.ibb.co/5FBTjbr/error-message.jpg

Also, after the script is running, how do you save the rendered video as an MP4 file? I ran it through AVIDemux to save it as an MP4, but I'm wondering how that's handled by someone who knows what they're doing.

714 (edited by ireactions 2023-04-09 12:10:32)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

I've been continuing with Hybrid... but I'm getting some odd results with NNEDI3. That combined with the grain settings is adding a lot of ugly textures, so I'm just going back to NNEDI3 without grain.

I found this additional Hybrid QTGMC menu in the denoise section:

https://i.ibb.co/XyRfzSs/hybrid.jpg

I wonder if pneumatic would have any advice.

I'm also wondering if pneumatic might advise me on h.264 settings:

https://i.ibb.co/jWwH6VF/h264.jpg

And I'm curious if the default NNEDI3 settings should be adjusted:

https://i.ibb.co/mB6sN3M/nnedi3.jpg

I recognize that Avisynth+ is superior, but I clearly don't understand it well enough yet to follow the directions and fill in any gaps.

715 (edited by pneumatic 2023-04-10 00:48:21)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

ireactions wrote:

Gives me this error:
https://i.ibb.co/5FBTjbr/error-message.jpg

Ah yes, it's another dependency I forgot about - sorry.

The dll is here and needs to go in C:\Windows\System32 , as per instructions under "runtime dependencies" here.   It's not actually an Avisynth dll, it's a third party dll that does fast fourier transforms used by QTGMC.   DLL's are basically precompiled C code that any application can use by calling the precompiled functions inside them.   

ireactions wrote:

Also, after the script is running, how do you save the rendered video as an MP4 file?

I'm using ffmpeg at the windows command prompt, eg. paste the following into command prompt:

"c:\program files\ffmpeg\bin\ffmpeg.exe" -color_range 1 -colorspace 1 -color_primaries 1 -i "C:\YourAvisynthScript.avs" -c:v libx264 -tune film -b:v 8000000 -pix_fmt yuv420p -color_range 1 -colorspace 1 -color_primaries 1 "C:\Detelecined.mkv"

'-colorspace 1' and '-color_primaries 1' mean HD colourimetry - use this if you upscaled to HD in your avisynth script.   SD colourimetry is 6 for both.   Note there are two instances to set in the above line - the first tells ffmpeg what the colourimetry of the input file is (i.e what the Avisynth script is outputting) and the second is what colourimetry ffmpeg should use for the output mp4 file.

'8000000' is the bit rate in bits per second (8mbps)


ireactions wrote:

I'm also wondering if pneumatic might advise me on h.264 settings:
https://i.ibb.co/jWwH6VF/h264.jpg

It's a bit hard to say as Hybrid seems to be using x264.exe which is a different package to ffmpeg.exe with different naming conventions.   I can see the tune is set to grain - I'd set that to film (unless you want it to preserve more grain?  I remember trying the grain preset on an episode of Oz some time ago and didn't like the way it made grain look more chunky, but maybe it was because I didn't set the bitrate high enough - 4mbps).   Underneath I can see it's got '--colormatrix bt470bg' which means SD PAL colourimetry which would produce wrong colours if the output file is HD.  NTSC colours are 170M, HD is 709.


ireactions wrote:

And I'm curious if the default NNEDI3 settings should be adjusted:
https://i.ibb.co/mB6sN3M/nnedi3.jpg

It seems ok.

716 (edited by pneumatic 2023-04-09 22:51:17)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

For future reference all TIVTC and QTGMC dependencies are in a zip file here & I've updated all previous references in this thread to point to it as well.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

It's working! Thanks, pneumatic.

I added the AddGrainC plugin and now I'm making various tweaks in pneumatic's script (and using Hybrid to give me the command lines needed to add into the file). I think it'd be nice to add in some film grain to fill in the blurriness and also up the contrast and saturation so that the episodes look a little less like the faded VHS cassettes they are and a little closer to the digital videotape look of Season 2 (although it'll still fall very short). Not every episode is in the same state: "Summer of Love" seems really blurry but pretty well-saturated; "Prince of Wails" seems a little undersaturated but pretty sharp; "Last Days" is downright muddy in its blurriness. I'm making those tweaks on an episode to episode basis with some thought to RussianCabbieLotteryFan's tastes for more colour.

I'm also experimenting with Lanczos4Resize or nnedi3 to bring it to a 1080p container. Which one would pneumatic use?

I know the resulting file won't be 1080p quality. But as pneumatic said earlier, it's worthwhile to avoid individual TVs or media players adding any more scaling artifacts to files that are already not in great shape. pneumatic has made them look far better than they ever have.

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

ireactions wrote:

I'm also experimenting with Lanczos4Resize or nnedi3 to bring it to a 1080p container. Which one would pneumatic use?

I'm using Lanczos or Spline36 only because my HTPC isn't powerful enough to do TIVTC + QTGMC repair + nnedi3 upscaling in realtime.    If you have the power to spare you might as well use nnedi3 as its very good at antialiasing high contrast edges - https://imgsli.com/MTU5MTAz

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

pneumatic wrote:
ireactions wrote:

I'm also experimenting with Lanczos4Resize or nnedi3 to bring it to a 1080p container. Which one would pneumatic use?

I'm using Lanczos or Spline36 only because my HTPC isn't powerful enough to do TIVTC + QTGMC repair + nnedi3 upscaling in realtime.    If you have the power to spare you might as well use nnedi3 as its very good at antialiasing high contrast edges - https://imgsli.com/MTU5MTAz

Huge difference!

720 (edited by RussianCabbie_Lotteryfan 2023-04-11 18:07:26)

Re: Sliders DVD Releases (Universal, Mill Creek, SD blu-ray, Restoration)

ireactions wrote:

It's working! Thanks, pneumatic.

I added the AddGrainC plugin and now I'm making various tweaks in pneumatic's script (and using Hybrid to give me the command lines needed to add into the file). I think it'd be nice to add in some film grain to fill in the blurriness and also up the contrast and saturation so that the episodes look a little less like the faded VHS cassettes they are and a little closer to the digital videotape look of Season 2 (although it'll still fall very short). Not every episode is in the same state: "Summer of Love" seems really blurry but pretty well-saturated; "Prince of Wails" seems a little undersaturated but pretty sharp; "Last Days" is downright muddy in its blurriness. I'm making those tweaks on an episode to episode basis with some thought to RussianCabbieLotteryFan's tastes for more colour.

I'm also experimenting with Lanczos4Resize or nnedi3 to bring it to a 1080p container. Which one would pneumatic use?

I know the resulting file won't be 1080p quality. But as pneumatic said earlier, it's worthwhile to avoid individual TVs or media players adding any more scaling artifacts to files that are already not in great shape. pneumatic has made them look far better than they ever have.

thank you!  i've spent the last five minutes trying to find an emoticon to express my gratitude on that (but having trouble finding a small one).