Software Functions
SightLine Application's Video Processing software is the foundation for all of our video processors. The software library provides a powerful suite of functions critical to advanced camera system integrators. Flexible configuration of functions allows integrators to optimize their system for a wide variety of Intelligence, Surveillance, and Reconnaissance (ISR), Inspection, and other applications.
- Software Overview presents a summary of SightLine’s suite of software functions.
Scene and Object Tracking
Robust hands-free tracking of scene and designated objects. Onboard tracker provides low latency solution needed for nimble pointing control systems. Tried and tested, our trackers have over one million flight hours logged. They are highly configurable to meet mission requirements and include advanced configuration options.
- Scene Tracking: Powerful, intuitive mode to keep the entire scene in the frame using scene translation
telemetry. Works well as a base mode and fallback from object trackers. - Vehicle and Stationary Modes: Solid designated tracks on moving or static objects. Advanced image
analysis and motion estimation isolate tracked object from background for robust tracking even when
view of tracked object is temporarily blocked. - No Registration Mode: Used to track objects when there are no background features (for example: trying
to track an airplane against a blue sky). - Track Box Size, Nudge, Track through Zoom and Rotation: Operational and setting flexibility for
improved tracker performance. - Acquisition Assist: Automatically searches the area around an initial track location to find the optimal
track box size and position. Allows operator to just get ‘close’ and still get great tracking results. - Intelligent Assist: Automatically reinitialize (and optimize track box location and size) on targets when
changes in the target appearance are detected. - Detection Track Cueing: Used to transition a detection to a primary track (example: aerial MTI
detection to vehicle object track). - To address demand for a lower price-point option, Tracker-Only options use the same proven tracking
algorithms as the full-featured SLA, offered with limitations on other functions.
Detection
Multiple detection modes provide important situational awareness information and provide fast feedback essential for effective track initialization on moving objects.
- Detect very small moving targets
- Simple track selection method – provided detection telemetry allows automatic track initialization
- Three MTI modes support a range of applications
- Color-based (histogram) anomaly detection algorithm helps find objects that are different than the scene background
- Radiometric detection finds objects within desired thermal range
Telemetry Data
Tracker functions and MTI provide low latency track location data needed to facilitate gimbal pointing.
- Telemetry updates at 30 Hz.
- Sub-pixel accuracy feedback for accurate gimbal pointing.
- Data is provided from object tracking, scene steering, and MTI objects for best system performance support.
- Temperature data from radiometric cameras.
- Focus telemetry for autofocus implementations.
Focus Telemetry
Algorithm calculates a focus metric which has a clear peak value when the scene is in the best focus.
- Focus metric is provided as telemetry data for customer autofocus implementations (focus motor control loop to optimize its setting)
- For some cameras and lenses, SightLine has included a complete autofocus control loop functionality, providing direct serial lens control for those devices
Precision Landing
The Landing Aid supports autonomous landing operations by automatically finding and tracking an easy to place landing pattern. It provides pattern position, range, and relative azimuth angle to the vehicle autonomous landing logic.
- Scalable landing pattern supports landing operations over a wide variety of ranges and approach angles.
- Independent of GPS, enables accurate landing in GPS-denied environments.
- Range and angle estimate replaces expensive laser based ranging systems. Inherently eye safe.
- Integration of telemetry data with Autopilots to provide autonomous guidance
- Sample source code available to ease customer integration. Piccolo, Pixhawk PX4 and APM functionality
has been demonstrated.
HD Video Input
Providing solutions that support both HD camera inputs and outputs is required for today’s camera systems. Both of SightLine’s hardware solutions and all of our software functions support HD video processing.
- 4K – (2160P) on 4000-OEM.
- 2x 1080p/30 processing / encoding on SLA-3000.
- 720p/30 input and up to 720p/25 output on SLA-1500.
- Adaptors for HD cameras, HDSDI, HDMI, CL, LVDS, etc.
Render Functions
IP Encoding
Encoded video ethernet streaming supports IP radio options and reduces system bandwidth. Controllable tradeoff of video compression frame rate, and down sampling to meet bandwidth requirements with best possible imagery and system flexibility.
- H.264 (1500-OEM, 3000-OEM, 4000-OEM)
- H.265 (4000-OEM)
- MPEG4/M-JPEG
- MPEG2 TS/RTP encapsulation
- UDP/TCP
- RTSP
- Unicast/Multicast/Broadcast
Stabilization and Roll Correction
The starting point for all video processing improvements is our frame to frame registration measurement. Registration provides the information to correct the output video frame to remove frame to frame motion (vibration) and dramatically improve video.
- Absolutely critical for camera systems on moving vehicles (air, ground, sea) and desired zoom < 10 deg FOV
- Accurate frame to frame change detection and correction
- Remove camera system jitter (frame to frame movement). Excellent for high frequency (approaching 30 Hz), lower amplitude noise rejection.
Registration also provides ability to correct camera frame to frame rotation, further improving the user experience.
- Correction of platform roll motion
- Important for cameras systems on aerostats, towers, UAS, and marine vehicles
- Critical for roll/nod gimbal configurations
- Provides 3 axis performance in a 2 axis design, saving complexity, weight, and expense
- Integrators can feed-forward camera roll commands to counteract known platform rotation.
KLV Metadata
- KLV metadata is generated in accordance with MISB standards 0102.10, 0601.7, 0603.2, 0604.3, and 0903.3.
- Multiple simultaneous streams +KLV on SLA-3000 and 4000-OEM
- Multiple sources include: C2, NMEA, pass-through
- Metadata on VBI lines for KLV over analog links.
High Bit Depth
Enables full pixel depth functionality (beyond 8 bit) which is critical for functions such as DPR/NUC and improves performance of detection and most enhancement functions. Enables recording of absolute amplitude snapshots.
Enhancement
A range of functions are provided to enable enhancement options to optimize video for the needs of each application.
- Contrast Limited Adaptive Histogram Equalization (CLAHE)
- Local Area Processing (LAP)
- De-noising
- Edge sharpening
- Mitigating scintillation effects
- False color schemes
- Extended 14 bit-depth processing
- Digital zoom
NUC and DPR
Dead Pixel Removal (DPR) and Non-Uniformity Correction (NUC) add capabilities to IR cameras, removing the need for a dedicated NUC/DPR board.
Recording / Snapshot
Onboard recording on 1500-OEM. Interface board SD card for 3000.
- H.264 video to local SD card or remote FTP.
- Full resolution snapshot recording. JPEG snapshots with EXIF data.
- Full pixel depth data .PNG snapshots with metadata (for radiometric data access).
- Telemetry data recording.
Blending
Multi-spectral blending, 3000-OEM only, provides for best situational awareness.
HD Video Output
Providing solutions that support both HD camera inputs and outputs is required for today’s camera systems. All of SightLine’s hardware solutions and software functions support HD video processing.
- 4K HDMI HD video output on 4000-OEM.
- 2x 1080p/30 processing / encoding on SLA 3000.
- 720p/30 input and up to 720p/25 output on SLA-1500.
- Adaptors for HD cameras, HDSDI, HDMI, CL, LVDS, etc.
On Screen Display (OSD)
- Protocol command OSD support to add text, shapes, and custom reticle overlays
- Bench tool to help customers use OSD primitives to build a custom OSD design.
- Customer logo placement.
- On 3000, ARM process support allows full OSD control to customer to design complex OSD implementations
Multi Camera Video Presentation
Flexible display options for optimized presentation and transmission and display of multiple video
streams.
- Full screen (switched video)
- Picture in Picture (selectable PiP window location/size) – 3000
- Split screen – 3000
- Multiple IP Streams – individual camera streams or combination of above – 3000
- Image Blending. Multi-spectral inputs can be blended into a single image with false color to bring out
the best of each spectral image – 3000
Customer Functions
Customer Developed Scripts
To best support customers ability to combine software functions to best meet their system performance objectives, SightLine processors support Lua based scripting functionality. Some examples that could be implemented using these scripts:
- After detecting a person in a defined area of a thermal image, automatically take a snapshot and append vehicle GPS and timestamp data.
Customer Developed ARM Applications
Both processors include an ARM processor which is very lightly used by SightLine. It is available to perform customer functions within their system architecture and allows access to serial ports, GPIO, etc. SigthLine provides sample code for a number of applications. Uses include:
- Autopilot interface for landing aid functionality.
- Gimbal control
- Protocol translation – system protocol to SightLine and vice versa