RTSP streaming

Products: Video Capture SDK .Net, Video Edit SDK .Net, Media Blocks SDK .Net

The SDK offers the capability to stream video and audio using the Real-Time Streaming Protocol (RTSP). It supports H264, a widely-used video compression standard, ensuring high-quality video streaming. Alongside, it also supports the Advanced Audio Coding (AAC) codec, known for its efficiency in delivering high-quality audio. This combination of RTSP with H264 and AAC codecs makes the SDK highly effective for applications requiring reliable and high-definition audiovisual streaming. It's particularly suited for surveillance, live broadcasting, and video conferencing solutions, where both clarity of video and audio are paramount.

Cross-platform RTSP output (X-engines: VideoCaptureCoreX, VideoEditCoreX, Media Blocks)

RTSPServerOutput

The RTSPServerOutput class serves as the configuration interface for video and audio encoding options in the RTSP server. It implements both IVideoEditXBaseOutput and IVideoCaptureXBaseOutput interfaces, making it compatible with both video editing and capture scenarios.

Key features:

• Provides access to RTSP server settings through the Settings property
• Offers a comprehensive list of supported video encoders including:
  • Hardware-accelerated options (NVENC, QSV, AMF)
  • Software-based options (OpenH264, VP8, VP9)
  • Platform-specific options (MF HEVC for Windows)
• Supports multiple audio encoding formats:
  • AAC variants (VO-AAC, AVENC AAC, MF AAC)
  • MP3 and OPUS formats

RTSPServerSettings

The RTSPServerSettings class manages the configuration of the RTSP server instance. It encapsulates all necessary parameters for setting up and running an RTSP server.

Important properties:

• Network Configuration:

  • Port: Server listening port (default: 8554)
  • Address: Server IP address (default: "127.0.0.1")
  • Point: Stream endpoint path (default: "/live")

• Stream Configuration:

  • VideoEncoder: Configures video encoding settings
  • AudioEncoder: Configures audio encoding settings
  • Latency: Controls streaming delay (default: 250ms)

• Authentication:

  • Username: Optional authentication username
  • Password: Optional authentication password

• Server Identity:

  • Name: Server name in stream metadata
  • Description: Server description in stream metadata

The class provides a convenient URL property that constructs the complete RTSP URL from the configured settings.

RTSPServerBlock

The RTSPServerBlock class represents the actual streaming server implementation, handling the media pipeline and stream processing. It inherits from MediaBlock and implements IMediaBlockInternals and IMediaBlockSink.

Key capabilities:

• Manages separate video and audio input pads for stream ingestion
• Handles GStreamer pipeline integration for media processing
• Provides runtime availability checking through IsAvailable()
• Implements proper resource cleanup through the IDisposable pattern

Usage Examples

Basic Server Setup

// Create encoder settings
var videoEncoder = new OpenH264EncoderSettings();
var audioEncoder = new VOAACEncoderSettings();

// Configure server settings
var settings = new RTSPServerSettings(videoEncoder, audioEncoder)
{
    Port = 8554,
    Address = "0.0.0.0",  // Listen on all interfaces
    Point = "/stream"
};

// Create and initialize server
var rtspOutput = new RTSPServerOutput(settings);

// Add output to the capture engine
videoCapture.Outputs_Add(rtspOutput, true); // videoCapture is an instance of VideoCaptureCoreX

// or set the output for the edit engine
videoEdit.Output_Format = rtspOutput; // videoEdit is an instance of VideoEditCoreX

Media Blocks sample

// Create encoder settings
IVideoEncoder videoEncoder = new OpenH264EncoderSettings();
IAudioEncoder audioEncoder = new VOAACEncoderSettings();

// Create server block
var rtspSink = new RTSPServerBlock(new RTSPServerSettings(new Uri("rtsp://127.0.0.1/live"), videoEncoder, audioEncoder));

// Connect video and audio streams to the server block
pipeline.Connect(videoSource.Output, rtspSink.VideoInput);
pipeline.Connect(audioSource.Output, rtspSink.AudioInput);                

Advanced Configuration

// Create server with custom URI
var uri = new Uri("rtsp://streaming.example.com:8555/live");
var settings = new RTSPServerSettings(uri, videoEncoder, audioEncoder)
{
    Username = "admin",
    Password = "secure123",
    Latency = TimeSpan.FromMilliseconds(500),
    Description = "Production Stream Server"
};

Best Practices

1. Encoder Selection: Choose encoders based on available hardware acceleration:

   • NVENC for NVIDIA GPUs
   • QSV for Intel processors
   • AMF for AMD graphics
   • OpenH264 for software fallback

2. Latency Configuration: Balance between stream smoothness and delay:

   • Lower values (100-200ms) for interactive scenarios
   • Higher values (500ms+) for better stream stability

3. Network Configuration: Consider security implications:

   • Use authentication for production environments
   • Bind to specific interfaces when possible
   • Configure firewalls to allow only necessary ports

4. Resource Management: Always properly dispose of server instances:

   • Use using statements or explicit Dispose() calls
   • Clean up both video and audio resources
   • Handle pipeline shutdown gracefully

Error Handling

The implementation includes several error handling mechanisms:

1. Availability checking through RTSPServerBlock.IsAvailable()
2. Build status verification in pipeline setup
3. Pad connection validation for both audio and video streams
4. Resource cleanup protection through proper disposal patterns

Performance Considerations

1. Encoder Selection Impact:

   • Hardware encoders provide better performance but require compatible hardware
   • Software encoders offer wider compatibility but higher CPU usage

2. Latency Trade-offs:

   • Lower latency may increase network bandwidth requirements
   • Higher latency provides better buffer against network jitter

3. Resource Usage:

   • Monitor CPU usage with software encoders
   • Watch memory usage with multiple simultaneous streams
   • Consider network bandwidth requirements for different quality settings

Windows-only RTSP output (VideoCaptureCore and VideoEditCore engines)

Sample code

Set the Network_Streaming_Enabled property to true to enable network streaming.

VideoCapture1.Network_Streaming_Enabled = true;

Enable audio streaming.

VideoCapture1.Network_Streaming_Audio_Enabled = true;

Set RTSP streaming format.

VideoCapture1.Network_Streaming_Format = NetworkStreamingFormat.RTSP_H264_AAC_SW;

Create MP4 output settings that contain H264 and AAC settings. MP4 file will not be created; we are using this class only to get H264 and AAC settings.

var mp4Output = new MP4Output();

Set H264 and AAC settings.

// ... 

Set the output.

VideoCapture1.Network_Streaming_Output = mp4Output;

Set the URL.

VideoCapture1.Network_Streaming_URL = "rtsp://localhost:5554/vfstream";

---

Visit our GitHub page to get more code samples.