diff --git a/README.md b/README.md new file mode 100644 index 0000000..0f07633 --- /dev/null +++ b/README.md @@ -0,0 +1,62 @@ +# README +## NVIDIA VideoEffects SDK: API Source Code and Sample Applications + +NVIDIA VideoEffects SDK is an SDK for enhancing and applying filters to videos at real-time. The SDK is powered by NVIDIA graphics processing units (GPUs) with Tensor Cores, and as a result, the algorithm throughput is greatly accelerated, and latency is reduced. + +NVIDIA VideoEffects SDK has the following AI features: + +- **Encoder Artifact Reduction**, which reduces the blocky and noisy artifacts from an encoded video while preserving the details of the original video. +- **Super Resolution**, which upscales a video while also reducing the blocky and noisy artifacts. It can enhance the details and sharpen the output while simultaneously preserving the content. This is suitable for upscaling lossy content. +- **Upscale**, which is a very fast and light-weight method for upscaling an input video. It also provides a sharpening parameter to sharpen the resulting output. This feature can be optionally pipelined with the encoder artifact reduction feature to enhance the scale while reducing the video artifacts. + +NVIDIA VideoEffects SDK provides two sample applications that demonstrate the features listed above in real time by using offline videos. +- **VideoEffects App**, which is a sample app that can invoke each feature individually. +- **UpscalePipeline App**, which is a sample app that pipelines the Encoder Artifact Reduction feature with the Upscale feature. + +All features in the VideoEffects SDK support 720p and 1080p as input resolutions. These are the scaling factors supported by the Super Resolution feature: +- **720p inputs** can be scaled by a factor of 1.5x or 2x. +- **1080p inputs** can be scaled by a factor of 1.33x or 2x. + +Additionally, the Upscale feature supports any input resolution, and the following scaling factors: +- 1.33x, 1.5x, 2x or 3x + +NVIDIA VideoEffects SDK is distributed in the following parts: + +- This open source repository that includes the [SDK API and proxy linking source code](https://github.com/NVIDIA/BROADCAST-VFX-SDK/tree/master/nvvfx), and [sample applications and their dependency libraries](https://github.com/NVIDIA/BROADCAST-VFX-SDK/tree/master/samples). +- An installer hosted on [RTX broadcast engine developer page](https://developer.nvidia.com/rtx-broadcast-engine) that installs the SDK DLLs, the models, and the SDK dependency libraries. + +Please refer to [SDK programming guide](https://github.com/NVIDIA/BROADCAST-VFX-SDK/blob/master/docs/NVIDIA%20Video%20Effects%20SDK%20Programming%20Guide.pdf) for configuring and integrating the SDK, compiling and running the sample applications. + +## System requirements +The SDK is supported on NVIDIA GPUs that are based on the NVIDIA® Turing™ architecture. Although the SDK can run on Turing™ GPUs without Tensor Cores, it is optimized for much higher performance on GPUs with Tensor Cores. + +* Windows OS supported: 64-bit Windows 10 +* Microsoft Visual Studio: 2017 (MSVC15.0) or later +* CMake: v3.12 or later +* NVIDIA Graphics Driver for Windows: 455.57 or later + +## NVIDIA Branding Guidelines +If you integrate an NVIDIA Broadcast Engine SDK within your product, please follow the required branding guidelines that are available [here]( +https://nvidia.frontify.com/d/uAobRitG8H8B) + +## Compiling the sample apps + +### Steps + +The open source repository includes the source code to build the sample applications, and a proxy file NVVideoEffectsProxy.cpp to enable compilation without explicitly linking against the SDK DLL. + +**Note: To download the models and runtime dependencies required by the features, you need to run the [SDK Installer](https://developer.nvidia.com/rtx-broadcast-engine).** + +1. In the root folder of the downloaded source code, start the CMake GUI and specify the source folder and a build folder for the binary files. +* For the source folder, ensure that the path ends in OSS. +* For the build folder, ensure that the path ends in OSS/build. +2. Use CMake to configure and generate the Visual Studio solution file. +* Click Configure. +* When prompted to confirm that CMake can create the build folder, click OK. +* Select Visual Studio for the generator and x64 for the platform. +* To complete configuring the Visual Studio solution file, click Finish. +* To generate the Visual Studio Solution file, click Generate. +* Verify that the build folder contains the NvVideoEffects_SDK.sln file. +3. Use Visual Studio to generate the application binary .exe file from the NvVideoEffects_SDK.sln file. +* In CMake, to open Visual Studio, click Open Project. +* In Visual Studio, select Build > Build Solution. diff --git a/docs/NVIDIA Video Effects SDK Programming Guide.pdf b/docs/NVIDIA Video Effects SDK Programming Guide.pdf index 6cdd357..d196dd0 100644 Binary files a/docs/NVIDIA Video Effects SDK Programming Guide.pdf and b/docs/NVIDIA Video Effects SDK Programming Guide.pdf differ diff --git a/samples/UpscalePipelineApp/UpscalePipeline.cpp b/samples/UpscalePipelineApp/UpscalePipeline.cpp index c28c24e..7715914 100644 --- a/samples/UpscalePipelineApp/UpscalePipeline.cpp +++ b/samples/UpscalePipelineApp/UpscalePipeline.cpp @@ -59,7 +59,7 @@ bool FLAG_debug = false, FLAG_progress = false; int FLAG_resolution = 0, FLAG_arStrength = 0; -float FLAG_upscaleStrength = 0.f; +float FLAG_upscaleStrength = 0.2f; std::string FLAG_codec = "H264", FLAG_inFile, FLAG_outFile, diff --git a/samples/UpscalePipelineApp/UpscalePipelineApp.exe b/samples/UpscalePipelineApp/UpscalePipelineApp.exe index e0b6483..7fc0e56 100644 Binary files a/samples/UpscalePipelineApp/UpscalePipelineApp.exe and b/samples/UpscalePipelineApp/UpscalePipelineApp.exe differ diff --git a/samples/UpscalePipelineApp/run.bat b/samples/UpscalePipelineApp/run.bat index 4b15000..2aa192f 100644 --- a/samples/UpscalePipelineApp/run.bat +++ b/samples/UpscalePipelineApp/run.bat @@ -1,6 +1,6 @@ SETLOCAL SET PATH=%PATH%;..\external\opencv\bin; REM Use --show to show the output in a window or use --out_file= to write output to file -UpscalePipelineApp.exe --in_file=..\input\input2.png --ar_strength=0 --upscale_strength=0 --resolution=1080 --show --out_file=ar_sr_0.png -UpscalePipelineApp.exe --in_file=..\input\input2.png --ar_strength=0 --upscale_strength=1 --resolution=1080 --show --out_file=ar_sr_1.png +UpscalePipelineApp.exe --in_file=..\input\input1.jpg --ar_strength=0 --upscale_strength=0 --resolution=1080 --show --out_file=ar_sr_0.png +UpscalePipelineApp.exe --in_file=..\input\input1.jpg --ar_strength=0 --upscale_strength=1 --resolution=1080 --show --out_file=ar_sr_1.png diff --git a/samples/VideoEffectsApp/VideoEffectsApp.cpp b/samples/VideoEffectsApp/VideoEffectsApp.cpp index 941bc7f..a8cde32 100644 --- a/samples/VideoEffectsApp/VideoEffectsApp.cpp +++ b/samples/VideoEffectsApp/VideoEffectsApp.cpp @@ -488,7 +488,7 @@ FXApp::Err FXApp::processImage(const char *inFile, const char *outFile) { BAIL_IF_ERR(vfxErr = NvVFX_SetCudaStream(_eff, NVVFX_CUDA_STREAM, stream)); if (!strcmp(_effectName, NVVFX_FX_ARTIFACT_REDUCTION)) { BAIL_IF_ERR(vfxErr = NvVFX_SetU32(_eff, NVVFX_STRENGTH, (unsigned int)FLAG_strength)); - } else if (!strcasecmp(_effectName, NVVFX_FX_SUPER_RES)) { + } else if (!strcmp(_effectName, NVVFX_FX_SUPER_RES)) { BAIL_IF_ERR(vfxErr = NvVFX_SetU32(_eff, NVVFX_STRENGTH, (unsigned int)FLAG_strength)); } @@ -548,7 +548,7 @@ FXApp::Err FXApp::processMovie(const char *inFile, const char *outFile) { BAIL_IF_ERR(vfxErr = NvVFX_SetCudaStream(_eff, NVVFX_CUDA_STREAM, stream)); if (!strcmp(_effectName, NVVFX_FX_ARTIFACT_REDUCTION)) { BAIL_IF_ERR(vfxErr = NvVFX_SetU32(_eff, NVVFX_STRENGTH, (unsigned int)FLAG_strength)); - } else if (!strcasecmp(_effectName, NVVFX_FX_SUPER_RES)) { + } else if (!strcmp(_effectName, NVVFX_FX_SUPER_RES)) { BAIL_IF_ERR(vfxErr = NvVFX_SetU32(_eff, NVVFX_STRENGTH, (unsigned int)FLAG_strength)); } BAIL_IF_ERR(vfxErr = NvVFX_Load(_eff)); diff --git a/samples/VideoEffectsApp/VideoEffectsApp.exe b/samples/VideoEffectsApp/VideoEffectsApp.exe index d02645c..a4fd058 100644 Binary files a/samples/VideoEffectsApp/VideoEffectsApp.exe and b/samples/VideoEffectsApp/VideoEffectsApp.exe differ diff --git a/samples/VideoEffectsApp/run.bat b/samples/VideoEffectsApp/run.bat index e089849..9626a6b 100644 --- a/samples/VideoEffectsApp/run.bat +++ b/samples/VideoEffectsApp/run.bat @@ -3,5 +3,5 @@ SET PATH=%PATH%;..\external\opencv\bin; REM Use --show to show the output in a window or use --out_file= to write output to file VideoEffectsApp.exe --in_file=..\input\input1.jpg --out_file=ar_1.png --effect=ArtifactReduction --strength=1 --show VideoEffectsApp.exe --in_file=..\input\input1.jpg --out_file=ar_0.png --effect=ArtifactReduction --strength=0 --show -VideoEffectsApp.exe --in_file=..\input\input2.png --out_file=sr_0.png --effect=SuperRes --resolution=1080 --strength=0 --show -VideoEffectsApp.exe --in_file=..\input\input2.png --out_file=sr_1.png --effect=SuperRes --resolution=1080 --strength=1 --show \ No newline at end of file +VideoEffectsApp.exe --in_file=..\input\input2.jpg --out_file=sr_0.png --effect=SuperRes --resolution=2160 --strength=0 --show +VideoEffectsApp.exe --in_file=..\input\input2.jpg --out_file=sr_1.png --effect=SuperRes --resolution=2160 --strength=1 --show \ No newline at end of file diff --git a/samples/input/input1.jpg b/samples/input/input1.jpg index 1e0898a..e53bb56 100644 Binary files a/samples/input/input1.jpg and b/samples/input/input1.jpg differ diff --git a/samples/input/input2.jpg b/samples/input/input2.jpg new file mode 100644 index 0000000..964d9f5 Binary files /dev/null and b/samples/input/input2.jpg differ