Copyright	Will Thompson Iñaki García Etxebarria and Jonas Platte
License	LGPL-2.1
Maintainer	Iñaki García Etxebarria ([email protected])
Safe Haskell	None
Language	Haskell2010

GI.GstVideo.Objects.VideoDecoder

Contents

Exported types
Methods

Description

This base class is for video decoders turning encoded data into raw video frames.

The GstVideoDecoder base class and derived subclasses should cooperate as follows:

Configuration

Initially, GstVideoDecoder calls start when the decoder element is activated, which allows the subclass to perform any global setup.

GstVideoDecoder calls setFormat to inform the subclass of caps describing input video data that it is about to receive, including possibly configuration data. While unlikely, it might be called more than once, if changing input parameters require reconfiguration.
Incoming data buffers are processed as needed, described in Data Processing below.
GstVideoDecoder calls stop at end of all processing.

Data processing

The base class gathers input data, and optionally allows subclass to parse this into subsequently manageable chunks, typically corresponding to and referred to as 'frames'.

Each input frame is provided in turn to the subclass' handleFrame callback. The ownership of the frame is given to the handleFrame callback.
If codec processing results in decoded data, the subclass should call gstVideoDecoderFinishFrame to have decoded data pushed. downstream. Otherwise, the subclass must call gstVideoDecoderDropFrame, to allow the base class to do timestamp and offset tracking, and possibly to requeue the frame for a later attempt in the case of reverse playback.

Shutdown phase

The GstVideoDecoder class calls stop to inform the subclass that data parsing will be stopped.

Additional Notes

Seeking/Flushing

When the pipeline is seeked or otherwise flushed, the subclass is informed via a call to its reset callback, with the hard parameter set to true. This indicates the subclass should drop any internal data queues and timestamps and prepare for a fresh set of buffers to arrive for parsing and decoding.
End Of Stream
At end-of-stream, the subclass parse function may be called some final times with the at_eos parameter set to true, indicating that the element should not expect any more data to be arriving, and it should parse and remaining frames and call videoDecoderHaveFrame if possible.

The subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It also needs to provide information about the ouptput caps, when they are known. This may be when the base class calls the subclass' setFormat function, though it might be during decoding, before calling gstVideoDecoderFinishFrame. This is done via gstVideoDecoderSetOutputState

The subclass is also responsible for providing (presentation) timestamps (likely based on corresponding input ones). If that is not applicable or possible, the base class provides limited framerate based interpolation.

Similarly, the base class provides some limited (legacy) seeking support if specifically requested by the subclass, as full-fledged support should rather be left to upstream demuxer, parser or alike. This simple approach caters for seeking and duration reporting using estimated input bitrates. To enable it, a subclass should call gstVideoDecoderSetEstimateRate to enable handling of incoming byte-streams.

The base class provides some support for reverse playback, in particular in case incoming data is not packetized or upstream does not provide fragments on keyframe boundaries. However, the subclass should then be prepared for the parsing and frame processing stage to occur separately (in normal forward processing, the latter immediately follows the former), The subclass also needs to ensure the parsing stage properly marks keyframes, unless it knows the upstream elements will do so properly for incoming data.

The bare minimum that a functional subclass needs to implement is:

Provide pad templates
Inform the base class of output caps via gstVideoDecoderSetOutputState
Parse input data, if it is not considered packetized from upstream Data will be provided to parse which should invoke gstVideoDecoderAddToFrame and gstVideoDecoderHaveFrame to separate the data belonging to each video frame.
Accept data in handleFrame and provide decoded results to gstVideoDecoderFinishFrame, or call gstVideoDecoderDropFrame.

Synopsis

newtype VideoDecoder = VideoDecoder (ManagedPtr VideoDecoder)
class (GObject o, IsDescendantOf VideoDecoder o) => IsVideoDecoder o
toVideoDecoder :: (MonadIO m, IsVideoDecoder o) => o -> m VideoDecoder
noVideoDecoder :: Maybe VideoDecoder
videoDecoderAddToFrame :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> Int32 -> m ()
videoDecoderAllocateOutputBuffer :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> m Buffer
videoDecoderAllocateOutputFrame :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> VideoCodecFrame -> m FlowReturn
videoDecoderAllocateOutputFrameWithParams :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> VideoCodecFrame -> BufferPoolAcquireParams -> m FlowReturn
videoDecoderDropFrame :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> VideoCodecFrame -> m FlowReturn
videoDecoderFinishFrame :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> VideoCodecFrame -> m FlowReturn
videoDecoderGetAllocator :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> m (Allocator, AllocationParams)
videoDecoderGetBufferPool :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> m BufferPool
videoDecoderGetEstimateRate :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> m Int32
videoDecoderGetFrame :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> Int32 -> m VideoCodecFrame
videoDecoderGetFrames :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> m [VideoCodecFrame]
videoDecoderGetLatency :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> m (Word64, Word64)
videoDecoderGetMaxDecodeTime :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> VideoCodecFrame -> m Int64
videoDecoderGetMaxErrors :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> m Int32
videoDecoderGetNeedsFormat :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> m Bool
videoDecoderGetOldestFrame :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> m VideoCodecFrame
videoDecoderGetOutputState :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> m VideoCodecState
videoDecoderGetPacketized :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> m Bool
videoDecoderGetPendingFrameSize :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> m Word64
videoDecoderGetQosProportion :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> m Double
videoDecoderHaveFrame :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> m FlowReturn
videoDecoderMergeTags :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> Maybe TagList -> TagMergeMode -> m ()
videoDecoderNegotiate :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> m Bool
videoDecoderProxyGetcaps :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> Maybe Caps -> Maybe Caps -> m Caps
videoDecoderReleaseFrame :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> VideoCodecFrame -> m ()
videoDecoderSetEstimateRate :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> Bool -> m ()
videoDecoderSetInterlacedOutputState :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> VideoFormat -> VideoInterlaceMode -> Word32 -> Word32 -> Maybe VideoCodecState -> m VideoCodecState
videoDecoderSetLatency :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> Word64 -> Word64 -> m ()
videoDecoderSetMaxErrors :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> Int32 -> m ()
videoDecoderSetNeedsFormat :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> Bool -> m ()
videoDecoderSetOutputState :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> VideoFormat -> Word32 -> Word32 -> Maybe VideoCodecState -> m VideoCodecState
videoDecoderSetPacketized :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> Bool -> m ()
videoDecoderSetUseDefaultPadAcceptcaps :: (HasCallStack, MonadIO m, IsVideoDecoder a) => a -> Bool -> m ()

Exported types

newtype VideoDecoder Source #

Memory-managed wrapper type.

Constructors

VideoDecoder (ManagedPtr VideoDecoder)

Instances

GObject VideoDecoder Source #
Instance details Defined in GI.GstVideo.Objects.VideoDecoder Methods gobjectType :: IO GType #
HasParentTypes VideoDecoder Source #
Instance details Defined in GI.GstVideo.Objects.VideoDecoder
type ParentTypes VideoDecoder Source #
Instance details Defined in GI.GstVideo.Objects.VideoDecoder type ParentTypes VideoDecoder = Element ': (Object ': (Object ': ([] :: [Type])))

class (GObject o, IsDescendantOf VideoDecoder o) => IsVideoDecoder o Source #

Type class for types which can be safely cast to VideoDecoder, for instance with toVideoDecoder.

Instances

(GObject o, IsDescendantOf VideoDecoder o) => IsVideoDecoder o Source #
Instance details Defined in GI.GstVideo.Objects.VideoDecoder

toVideoDecoder :: (MonadIO m, IsVideoDecoder o) => o -> m VideoDecoder Source #

Cast to VideoDecoder, for types for which this is known to be safe. For general casts, use castTo.

noVideoDecoder :: Maybe VideoDecoder Source #

A convenience alias for Nothing :: Maybe VideoDecoder.

Methods

addToFrame

videoDecoderAddToFrame Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> Int32	`nBytes`: the number of bytes to add
-> m ()

Removes next nBytes of input data and adds it to currently parsed frame.

allocateOutputBuffer

videoDecoderAllocateOutputBuffer Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> m Buffer	Returns: allocated buffer, or NULL if no buffer could be allocated (e.g. when downstream is flushing or shutting down)

Helper function that allocates a buffer to hold a video frame for decoder's current VideoCodecState.

You should use videoDecoderAllocateOutputFrame instead of this function, if possible at all.

allocateOutputFrame

videoDecoderAllocateOutputFrame Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> VideoCodecFrame	`frame`: a `VideoCodecFrame`
-> m FlowReturn	Returns: `FlowReturnOk` if an output buffer could be allocated

Helper function that allocates a buffer to hold a video frame for decoder's current VideoCodecState. Subclass should already have configured video state and set src pad caps.

The buffer allocated here is owned by the frame and you should only keep references to the frame, not the buffer.

allocateOutputFrameWithParams

videoDecoderAllocateOutputFrameWithParams Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> VideoCodecFrame	`frame`: a `VideoCodecFrame`
-> BufferPoolAcquireParams	`params`: a `BufferPoolAcquireParams`
-> m FlowReturn	Returns: `FlowReturnOk` if an output buffer could be allocated

Same as gst_video_decoder_allocate_output_frame except it allows passing BufferPoolAcquireParams to the sub call gst_buffer_pool_acquire_buffer.

Since: 1.12

dropFrame

videoDecoderDropFrame Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`dec`: a `VideoDecoder`
-> VideoCodecFrame	`frame`: the `VideoCodecFrame` to drop
-> m FlowReturn	Returns: a `FlowReturn`, usually GST_FLOW_OK.

Similar to videoDecoderFinishFrame, but drops frame in any case and posts a QoS message with the frame's details on the bus. In any case, the frame is considered finished and released.

finishFrame

videoDecoderFinishFrame Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> VideoCodecFrame	`frame`: a decoded `VideoCodecFrame`
-> m FlowReturn	Returns: a `FlowReturn` resulting from sending data downstream

frame should have a valid decoded data buffer, whose metadata fields are then appropriately set according to frame data and pushed downstream. If no output data is provided, frame is considered skipped. In any case, the frame is considered finished and released.

After calling this function the output buffer of the frame is to be considered read-only. This function will also change the metadata of the buffer.

getAllocator

videoDecoderGetAllocator Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> m (Allocator, AllocationParams)

Lets VideoDecoder sub-classes to know the memory allocator used by the base class and its params.

Unref the allocator after use it.

getBufferPool

videoDecoderGetBufferPool Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> m BufferPool	Returns: the instance of the `BufferPool` used by the decoder; free it after use it

No description available in the introspection data.

getEstimateRate

videoDecoderGetEstimateRate Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`dec`: a `VideoDecoder`
-> m Int32	Returns: currently configured byte to time conversion setting

No description available in the introspection data.

getFrame

videoDecoderGetFrame Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> Int32	`frameNumber`: system_frame_number of a frame
-> m VideoCodecFrame	Returns: pending unfinished `VideoCodecFrame` identified by `frameNumber`.

Get a pending unfinished VideoCodecFrame

getFrames

videoDecoderGetFrames Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> m [VideoCodecFrame]	Returns: pending unfinished `VideoCodecFrame`.

Get all pending unfinished VideoCodecFrame

getLatency

videoDecoderGetLatency Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> m (Word64, Word64)

Query the configured decoder latency. Results will be returned via minLatency and maxLatency.

getMaxDecodeTime

videoDecoderGetMaxDecodeTime Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> VideoCodecFrame	`frame`: a `VideoCodecFrame`
-> m Int64	Returns: max decoding time.

Determines maximum possible decoding time for frame that will allow it to decode and arrive in time (as determined by QoS events). In particular, a negative result means decoding in time is no longer possible and should therefore occur as soon/skippy as possible.

getMaxErrors

videoDecoderGetMaxErrors Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`dec`: a `VideoDecoder`
-> m Int32	Returns: currently configured decoder tolerated error count.

No description available in the introspection data.

getNeedsFormat

videoDecoderGetNeedsFormat Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`dec`: a `VideoDecoder`
-> m Bool	Returns: `True` if required format handling is enabled.

Queries decoder required format handling.

Since: 1.4

getOldestFrame

videoDecoderGetOldestFrame Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> m VideoCodecFrame	Returns: oldest pending unfinished `VideoCodecFrame`.

Get the oldest pending unfinished VideoCodecFrame

getOutputState

videoDecoderGetOutputState Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> m VideoCodecState	Returns: `VideoCodecState` describing format of video data.

Get the VideoCodecState currently describing the output stream.

getPacketized

videoDecoderGetPacketized Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> m Bool	Returns: TRUE if input data is considered packetized.

Queries whether input data is considered packetized or not by the base class.

getPendingFrameSize

videoDecoderGetPendingFrameSize Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> m Word64	Returns: The number of bytes pending for the current frame

Returns the number of bytes previously added to the current frame by calling videoDecoderAddToFrame.

Since: 1.4

getQosProportion

videoDecoderGetQosProportion Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder` current QoS proportion, or `Nothing`
-> m Double	Returns: The current QoS proportion.

No description available in the introspection data.

Since: 1.0.3

haveFrame

videoDecoderHaveFrame Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> m FlowReturn	Returns: a `FlowReturn`

Gathers all data collected for currently parsed frame, gathers corresponding metadata and passes it along for further processing, i.e. handleFrame.

mergeTags

videoDecoderMergeTags Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> Maybe TagList	`tags`: a `TagList` to merge, or NULL to unset previously-set tags
-> TagMergeMode	`mode`: the `TagMergeMode` to use, usually `GST_TAG_MERGE_REPLACE`
-> m ()

Sets the audio decoder tags and how they should be merged with any upstream stream tags. This will override any tags previously-set with gst_audio_decoder_merge_tags().

Note that this is provided for convenience, and the subclass is not required to use this and can still do tag handling on its own.

MT safe.

negotiate

videoDecoderNegotiate Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> m Bool	Returns: `True` if the negotiation succeeded, else `False`.

Negotiate with downstream elements to currently configured VideoCodecState. Unmark GST_PAD_FLAG_NEED_RECONFIGURE in any case. But mark it again if negotiate fails.

proxyGetcaps

videoDecoderProxyGetcaps Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> Maybe Caps	`caps`: initial caps
-> Maybe Caps	`filter`: filter caps
-> m Caps	Returns: a `Caps` owned by caller

Returns caps that express caps (or sink template caps if caps == NULL) restricted to resolution/format/... combinations supported by downstream elements.

Since: 1.6

releaseFrame

videoDecoderReleaseFrame Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`dec`: a `VideoDecoder`
-> VideoCodecFrame	`frame`: the `VideoCodecFrame` to release
-> m ()

Similar to videoDecoderDropFrame, but simply releases frame without any processing other than removing it from list of pending frames, after which it is considered finished and released.

Since: 1.2.2

setEstimateRate

videoDecoderSetEstimateRate Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`dec`: a `VideoDecoder`
-> Bool	`enabled`: whether to enable byte to time conversion
-> m ()

Allows baseclass to perform byte to time estimated conversion.

setInterlacedOutputState

videoDecoderSetInterlacedOutputState Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> VideoFormat	`fmt`: a `VideoFormat`
-> VideoInterlaceMode	`mode`: A `VideoInterlaceMode`
-> Word32	`width`: The width in pixels
-> Word32	`height`: The height in pixels
-> Maybe VideoCodecState	`reference`: An optional reference `VideoCodecState`
-> m VideoCodecState	Returns: the newly configured output state.

Same as gst_video_decoder_set_output_state() but also allows you to also set the interlacing mode.

Since: 1.16.

setLatency

videoDecoderSetLatency Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> Word64	`minLatency`: minimum latency
-> Word64	`maxLatency`: maximum latency
-> m ()

Lets VideoDecoder sub-classes tell the baseclass what the decoder latency is. Will also post a LATENCY message on the bus so the pipeline can reconfigure its global latency.

setMaxErrors

videoDecoderSetMaxErrors Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`dec`: a `VideoDecoder`
-> Int32	`num`: max tolerated errors
-> m ()

Sets numbers of tolerated decoder errors, where a tolerated one is then only warned about, but more than tolerated will lead to fatal error. You can set -1 for never returning fatal errors. Default is set to GST_VIDEO_DECODER_MAX_ERRORS.

The '-1' option was added in 1.4

setNeedsFormat

videoDecoderSetNeedsFormat Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`dec`: a `VideoDecoder`
-> Bool	`enabled`: new state
-> m ()

Configures decoder format needs. If enabled, subclass needs to be negotiated with format caps before it can process any data. It will then never be handed any data before it has been configured. Otherwise, it might be handed data without having been configured and is then expected being able to do so either by default or based on the input data.

Since: 1.4

setOutputState

videoDecoderSetOutputState Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> VideoFormat	`fmt`: a `VideoFormat`
-> Word32	`width`: The width in pixels
-> Word32	`height`: The height in pixels
-> Maybe VideoCodecState	`reference`: An optional reference `VideoCodecState`
-> m VideoCodecState	Returns: the newly configured output state.

Creates a new VideoCodecState with the specified fmt, width and height as the output state for the decoder. Any previously set output state on decoder will be replaced by the newly created one.

If the subclass wishes to copy over existing fields (like pixel aspec ratio, or framerate) from an existing VideoCodecState, it can be provided as a reference.

If the subclass wishes to override some fields from the output state (like pixel-aspect-ratio or framerate) it can do so on the returned VideoCodecState.

The new output state will only take effect (set on pads and buffers) starting from the next call to gst_video_decoder_finish_frame().

setPacketized

videoDecoderSetPacketized Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> Bool	`packetized`: whether the input data should be considered as packetized.
-> m ()

Allows baseclass to consider input data as packetized or not. If the input is packetized, then the parse method will not be called.

setUseDefaultPadAcceptcaps

videoDecoderSetUseDefaultPadAcceptcaps Source #

Arguments

:: (HasCallStack, MonadIO m, IsVideoDecoder a)
=> a	`decoder`: a `VideoDecoder`
-> Bool	`use`: if the default pad accept-caps query handling should be used
-> m ()

Lets VideoDecoder sub-classes decide if they want the sink pad to use the default pad query handler to reply to accept-caps queries.

By setting this to true it is possible to further customize the default handler with GST_PAD_SET_ACCEPT_INTERSECT and GST_PAD_SET_ACCEPT_TEMPLATE

Since: 1.6