lavc: introduce a new decoding/encoding API with decoupled input/output

Until now, the decoding API was restricted to outputting 0 or 1 frames per input packet. It also enforces a somewhat rigid dataflow in general. This new API seeks to relax these restrictions by decoupling input and output. Instead of doing a single call on each decode step, which may consume the packet and may produce output, the new API requires the user to send input first, and then ask for output. For now, there are no codecs supporting this API. The API can work with codecs using the old API, and most code added here is to make them interoperate. The reverse is not possible, although for audio it might. From Libav commit 05f66706d182eb0c36af54d72614bf4c33e957a9.

wm4 committed 10 years ago

Commit 7fc329e1 parent a0b92786 files · +546 −8Browse files

Unified · Split

6 files

doc/APIchanges +5 −0
libavcodec/avcodec.h +238 −3
libavcodec/internal.h +13 −0
libavcodec/utils.c +287 −2
libavcodec/version.h +1 −1
libavformat/avformat.h +2 −2

doc/APIchanges+5−0

 API changes, most recent first:
+2016-xx-xx - xxxxxxx - lavc 57.37.100 - avcodec.h
+  Add a new audio/video encoding and decoding API with decoupled input
+  and output -- avcodec_send_packet(), avcodec_receive_frame(),
+  avcodec_send_frame() and avcodec_receive_packet().
 2016-xx-xx - xxxxxxx - lavc 57.15.0 - avcodec.h
   Add a new bitstream filtering API working with AVPackets.
   Deprecate the old bistream filtering API.

libavcodec/avcodec.h+238−3

@@ -72,6 +72,95 @@ · 71 unmodified lines
72	* @}	72	* @}
73	*/	73	*/
74		74
		75	/**
		76	* @ingroup libavc
		77	* @defgroup lavc_encdec send/receive encoding and decoding API overview
		78	* @{
		79	*
		80	* The avcodec_send_packet()/avcodec_receive_frame()/avcodec_send_frame()/
		81	* avcodec_receive_packet() functions provide an encode/decode API, which
		82	* decouples input and output.
		83	*
		84	* The API is very similar for encoding/decoding and audio/video, and works as
		85	* follows:
		86	* - Set up and open the AVCodecContext as usual.
		87	* - Send valid input:
		88	* - For decoding, call avcodec_send_packet() to give the decoder raw
		89	* compressed data in an AVPacket.
		90	* - For encoding, call avcodec_send_frame() to give the decoder an AVFrame
		91	* containing uncompressed audio or video.
		92	* In both cases, it is recommended that AVPackets and AVFrames are
		93	* refcounted, or libavcodec might have to copy the input data. (libavformat
		94	* always returns refcounted AVPackets, and av_frame_get_buffer() allocates
		95	* refcounted AVFrames.)
		96	* - Receive output in a loop. Periodically call one of the avcodec_receive_*()
		97	* functions and process their output:
		98	* - For decoding, call avcodec_receive_frame(). On success, it will return
		99	* an AVFrame containing uncompressed audio or video data.
		100	* - For encoding, call avcodec_receive_packet(). On success, it will return
		101	* an AVPacket with a compressed frame.
		102	* Repeat this call until it returns AVERROR(EAGAIN) or an error. The
		103	* AVERROR(EAGAIN) return value means that new input data is required to
		104	* return new output. In this case, continue with sending input. For each
		105	* input frame/packet, the codec will typically return 1 output frame/packet,
		106	* but it can also be 0 or more than 1.
		107	*
		108	* At the beginning of decoding or encoding, the codec might accept multiple
		109	* input frames/packets without returning a frame, until its internal buffers
		110	* are filled. This situation is handled transparently if you follow the steps
		111	* outlined above.
		112	*
		113	* End of stream situations. These require "flushing" (aka draining) the codec,
		114	* as the codec might buffer multiple frames or packets internally for
		115	* performance or out of necessity (consider B-frames).
		116	* This is handled as follows:
		117	* - Instead of valid input, send NULL to the avcodec_send_packet() (decoding)
		118	* or avcodec_send_frame() (encoding) functions. This will enter draining
		119	* mode.
		120	* - Call avcodec_receive_frame() (decoding) or avcodec_receive_packet()
		121	* (encoding) in a loop until AVERROR_EOF is returned. The functions will
		122	* not return AVERROR(EAGAIN), unless you forgot to enter draining mode.
		123	* - Before decoding can be resumed again, the codec has to be reset with
		124	* avcodec_flush_buffers().
		125	*
		126	* Using the API as outlined above is highly recommended. But it is also
		127	* possible to call functions outside of this rigid schema. For example, you can
		128	* call avcodec_send_packet() repeatedly without calling
		129	* avcodec_receive_frame(). In this case, avcodec_send_packet() will succeed
		130	* until the codec's internal buffer has been filled up (which is typically of
		131	* size 1 per output frame, after initial input), and then reject input with
		132	* AVERROR(EAGAIN). Once it starts rejecting input, you have no choice but to
		133	* read at least some output.
		134	*
		135	* Not all codecs will follow a rigid and predictable dataflow; the only
		136	* guarantee is that an AVERROR(EAGAIN) return value on a send/receive call on
		137	* one end implies that a receive/send call on the other end will succeed. In
		138	* general, no codec will permit unlimited buffering of input or output.
		139	*
		140	* This API replaces the following legacy functions:
		141	* - avcodec_decode_video2() and avcodec_decode_audio4():
		142	* Use avcodec_send_packet() to feed input to the decoder, then use
		143	* avcodec_receive_frame() to receive decoded frames after each packet.
		144	* Unlike with the old video decoding API, multiple frames might result from
		145	* a packet. For audio, splitting the input packet into frames by partially
		146	* decoding packets becomes transparent to the API user. You never need to
		147	* feed an AVPacket to the API twice.
		148	* Additionally, sending a flush/draining packet is required only once.
		149	* - avcodec_encode_video2()/avcodec_encode_audio2():
		150	* Use avcodec_send_frame() to feed input to the encoder, then use
		151	* avcodec_receive_packet() to receive encoded packets.
		152	* Providing user-allocated buffers for avcodec_receive_packet() is not
		153	* possible.
		154	* - The new API does not handle subtitles yet.
		155	*
		156	* Mixing new and old function calls on the same AVCodecContext is not allowed,
		157	* and will result in undefined behavior.
		158	*
		159	* Some codecs might require using the new API; using the old API will return
		160	* an error when calling it.
		161	* @}
		162	*/
		163
75	/**	164	/**
76	* @defgroup lavc_core Core functions/structures.	165	* @defgroup lavc_core Core functions/structures.
77	* @ingroup libavc	166	* @ingroup libavc
@@ -1740,7 +1829,7 @@ typedef struct AVCodecContext { · 1662 unmodified lines
1740	* the decoded frame is cropped before being output or lowres is enabled.	1829	* the decoded frame is cropped before being output or lowres is enabled.
1741	*	1830	*
1742	* @note Those field may not match the value of the last	1831	* @note Those field may not match the value of the last
1743	* AVFrame outputted by avcodec_decode_video2 due frame	1832	* AVFrame outputted by avcodec_receive_frame() due frame
1744	* reordering.	1833	* reordering.
1745	*	1834	*
1746	* - encoding: unused	1835	* - encoding: unused
@@ -1767,7 +1856,7 @@ typedef struct AVCodecContext { · 20 unmodified lines
1767	* May be overridden by the decoder if it knows better.	1856	* May be overridden by the decoder if it knows better.
1768	*	1857	*
1769	* @note This field may not match the value of the last	1858	* @note This field may not match the value of the last
1770	* AVFrame outputted by avcodec_decode_video2 due frame	1859	* AVFrame outputted by avcodec_receive_frame() due frame
1771	* reordering.	1860	* reordering.
1772	*	1861	*
1773	* - encoding: Set by user.	1862	* - encoding: Set by user.
@@ -2472,6 +2561,8 @@ typedef struct AVCodecContext { · 698 unmodified lines
2472	* Otherwise, the decoded frames must not be freed by the caller and are	2561	* Otherwise, the decoded frames must not be freed by the caller and are
2473	* only valid until the next decode call.	2562	* only valid until the next decode call.
2474	*	2563	*
		2564	* This is always automatically enabled if avcodec_receive_frame() is used.
		2565	*
2475	* - encoding: unused	2566	* - encoding: unused
2476	* - decoding: set by the caller before avcodec_open2().	2567	* - decoding: set by the caller before avcodec_open2().
2477	*/	2568	*/
@@ -3521,6 +3612,21 @@ typedef struct AVCodec { · 1043 unmodified lines
3521	int *got_packet_ptr);	3612	int *got_packet_ptr);
3522	int (decode)(AVCodecContext , void outdata, int outdata_size, AVPacket *avpkt);	3613	int (decode)(AVCodecContext , void outdata, int outdata_size, AVPacket *avpkt);
3523	int (close)(AVCodecContext );	3614	int (close)(AVCodecContext );
		3615	/**
		3616	* Decode/encode API with decoupled packet/frame dataflow. The API is the
		3617	* same as the avcodec_ prefixed APIs (avcodec_send_frame() etc.), except
		3618	* that:
		3619	* - never called if the codec is closed or the wrong type,
		3620	* - AVPacket parameter change side data is applied right before calling
		3621	* AVCodec->send_packet,
		3622	* - if AV_CODEC_CAP_DELAY is not set, drain packets or frames are never sent,
		3623	* - only one drain packet is ever passed down (until the next flush()),
		3624	* - a drain AVPacket is always NULL (no need to check for avpkt->size).
		3625	*/
		3626	int (send_frame)(AVCodecContext avctx, const AVFrame *frame);
		3627	int (send_packet)(AVCodecContext avctx, const AVPacket *avpkt);
		3628	int (receive_frame)(AVCodecContext avctx, AVFrame *frame);
		3629	int (receive_packet)(AVCodecContext avctx, AVPacket *avpkt);
3524	/**	3630	/**
3525	* Flush buffers.	3631	* Flush buffers.
3526	* Will be called when seeking	3632	* Will be called when seeking
@@ -4111,7 +4217,7 @@ int avcodec_parameters_to_context(AVCodecContext *codec, · 584 unmodified lines
4111	* @warning This function is not thread safe!	4217	* @warning This function is not thread safe!
4112	*	4218	*
4113	* @note Always call this function before using decoding routines (such as	4219	* @note Always call this function before using decoding routines (such as
4114	* @ref avcodec_decode_video2()).	4220	* @ref avcodec_receive_frame()).
4115	*	4221	*
4116	* @code	4222	* @code
4117	* avcodec_register_all();	4223	* avcodec_register_all();
@@ -4570,7 +4676,10 @@ enum AVChromaLocation avcodec_chroma_pos_to_enum(int xpos, int ypos); · 452 unmodified lines
4570	* @return A negative error code is returned if an error occurred during	4676	* @return A negative error code is returned if an error occurred during
4571	* decoding, otherwise the number of bytes consumed from the input	4677	* decoding, otherwise the number of bytes consumed from the input
4572	* AVPacket is returned.	4678	* AVPacket is returned.
		4679	*
		4680	* @deprecated Use avcodec_send_packet() and avcodec_receive_frame().
4573	*/	4681	*/
		4682	attribute_deprecated
4574	int avcodec_decode_audio4(AVCodecContext avctx, AVFrame frame,	4683	int avcodec_decode_audio4(AVCodecContext avctx, AVFrame frame,
4575	int got_frame_ptr, const AVPacket avpkt);	4684	int got_frame_ptr, const AVPacket avpkt);
4576		4685
@@ -4616,7 +4725,10 @@ int avcodec_decode_audio4(AVCodecContext avctx, AVFrame frame, · 39 unmodified lines
4616	* @param[in,out] got_picture_ptr Zero if no frame could be decompressed, otherwise, it is nonzero.	4725	* @param[in,out] got_picture_ptr Zero if no frame could be decompressed, otherwise, it is nonzero.
4617	* @return On error a negative value is returned, otherwise the number of bytes	4726	* @return On error a negative value is returned, otherwise the number of bytes
4618	* used or zero if no frame could be decompressed.	4727	* used or zero if no frame could be decompressed.
		4728	*
		4729	* @deprecated Use avcodec_send_packet() and avcodec_receive_frame().
4619	*/	4730	*/
		4731	attribute_deprecated
4620	int avcodec_decode_video2(AVCodecContext avctx, AVFrame picture,	4732	int avcodec_decode_video2(AVCodecContext avctx, AVFrame picture,
4621	int *got_picture_ptr,	4733	int *got_picture_ptr,
4622	const AVPacket *avpkt);	4734	const AVPacket *avpkt);
@@ -4652,6 +4764,129 @@ int avcodec_decode_subtitle2(AVCodecContext avctx, AVSubtitle sub, · 29 unmodified lines
4652	int *got_sub_ptr,	4764	int *got_sub_ptr,
4653	AVPacket *avpkt);	4765	AVPacket *avpkt);
4654		4766
		4767	/**
		4768	* Supply raw packet data as input to a decoder.
		4769	*
		4770	* Internally, this call will copy relevant AVCodecContext fields, which can
		4771	* influence decoding per-packet, and apply them when the packet is actually
		4772	* decoded. (For example AVCodecContext.skip_frame, which might direct the
		4773	* decoder to drop the frame contained by the packet sent with this function.)
		4774	*
		4775	* @warning The input buffer, avpkt->data must be AV_INPUT_BUFFER_PADDING_SIZE
		4776	* larger than the actual read bytes because some optimized bitstream
		4777	* readers read 32 or 64 bits at once and could read over the end.
		4778	*
		4779	* @warning Do not mix this API with the legacy API (like avcodec_decode_video2())
		4780	* on the same AVCodecContext. It will return unexpected results now
		4781	* or in future libavcodec versions.
		4782	*
		4783	* @note The AVCodecContext MUST have been opened with @ref avcodec_open2()
		4784	* before packets may be fed to the decoder.
		4785	*
		4786	* @param avctx codec context
		4787	* @param[in] avpkt The input AVPacket. Usually, this will be a single video
		4788	* frame, or several complete audio frames.
		4789	* Ownership of the packet remains with the caller, and the
		4790	* decoder will not write to the packet. The decoder may create
		4791	* a reference to the packet data (or copy it if the packet is
		4792	* not reference-counted).
		4793	* Unlike with older APIs, the packet is always fully consumed,
		4794	* and if it contains multiple frames (e.g. some audio codecs),
		4795	* will require you to call avcodec_receive_frame() multiple
		4796	* times afterwards before you can send a new packet.
		4797	* It can be NULL (or an AVPacket with data set to NULL and
		4798	* size set to 0); in this case, it is considered a flush
		4799	* packet, which signals the end of the stream. Sending the
		4800	* first flush packet will return success. Subsequent ones are
		4801	* unnecessary and will return AVERROR_EOF. If the decoder
		4802	* still has frames buffered, it will return them after sending
		4803	* a flush packet.
		4804	*
		4805	* @return 0 on success, otherwise negative error code:
		4806	* AVERROR(EAGAIN): input is not accepted right now - the packet must be
		4807	* resent after trying to read output
		4808	* AVERROR_EOF: the decoder has been flushed, and no new packets can
		4809	* be sent to it (also returned if more than 1 flush
		4810	* packet is sent)
		4811	* AVERROR(EINVAL): codec not opened, it is an encoder, or requires flush
		4812	* AVERROR(ENOMEM): failed to add packet to internal queue, or similar
		4813	* other errors: legitimate decoding errors
		4814	*/
		4815	int avcodec_send_packet(AVCodecContext avctx, const AVPacket avpkt);
		4816
		4817	/**
		4818	* Return decoded output data from a decoder.
		4819	*
		4820	* @param avctx codec context
		4821	* @param frame This will be set to a reference-counted video or audio
		4822	* frame (depending on the decoder type) allocated by the
		4823	* decoder. Note that the function will always call
		4824	* av_frame_unref(frame) before doing anything else.
		4825	*
		4826	* @return
		4827	* 0: success, a frame was returned
		4828	* AVERROR(EAGAIN): output is not available right now - user must try
		4829	* to send new input
		4830	* AVERROR_EOF: the decoder has been fully flushed, and there will be
		4831	* no more output frames
		4832	* AVERROR(EINVAL): codec not opened, or it is an encoder
		4833	* other negative values: legitimate decoding errors
		4834	*/
		4835	int avcodec_receive_frame(AVCodecContext avctx, AVFrame frame);
		4836
		4837	/**
		4838	* Supply a raw video or audio frame to the encoder. Use avcodec_receive_packet()
		4839	* to retrieve buffered output packets.
		4840	*
		4841	* @param avctx codec context
		4842	* @param[in] frame AVFrame containing the raw audio or video frame to be encoded.
		4843	* Ownership of the frame remains with the caller, and the
		4844	* encoder will not write to the frame. The encoder may create
		4845	* a reference to the frame data (or copy it if the frame is
		4846	* not reference-counted).
		4847	* It can be NULL, in which case it is considered a flush
		4848	* packet. This signals the end of the stream. If the encoder
		4849	* still has packets buffered, it will return them after this
		4850	* call. Once flushing mode has been entered, additional flush
		4851	* packets are ignored, and sending frames will return
		4852	* AVERROR_EOF.
		4853	*
		4854	* For audio:
		4855	* If AV_CODEC_CAP_VARIABLE_FRAME_SIZE is set, then each frame
		4856	* can have any number of samples.
		4857	* If it is not set, frame->nb_samples must be equal to
		4858	* avctx->frame_size for all frames except the last.
		4859	* The final frame may be smaller than avctx->frame_size.
		4860	* @return 0 on success, otherwise negative error code:
		4861	* AVERROR(EAGAIN): input is not accepted right now - the frame must be
		4862	* resent after trying to read output packets
		4863	* AVERROR_EOF: the encoder has been flushed, and no new frames can
		4864	* be sent to it
		4865	* AVERROR(EINVAL): codec not opened, refcounted_frames not set, it is a
		4866	* decoder, or requires flush
		4867	* AVERROR(ENOMEM): failed to add packet to internal queue, or similar
		4868	* other errors: legitimate decoding errors
		4869	*/
		4870	int avcodec_send_frame(AVCodecContext avctx, const AVFrame frame);
		4871
		4872	/**
		4873	* Read encoded data from the encoder.
		4874	*
		4875	* @param avctx codec context
		4876	* @param avpkt This will be set to a reference-counted packet allocated by the
		4877	* encoder. Note that the function will always call
		4878	* av_frame_unref(frame) before doing anything else.
		4879	* @return 0 on success, otherwise negative error code:
		4880	* AVERROR(EAGAIN): output is not available right now - user must try
		4881	* to send input
		4882	* AVERROR_EOF: the encoder has been fully flushed, and there will be
		4883	* no more output packets
		4884	* AVERROR(EINVAL): codec not opened, or it is an encoder
		4885	* other errors: legitimate decoding errors
		4886	*/
		4887	int avcodec_receive_packet(AVCodecContext avctx, AVPacket avpkt);
		4888
		4889
4655	/**	4890	/**
4656	* @defgroup lavc_parsing Frame parsing	4891	* @defgroup lavc_parsing Frame parsing
4657	* @{	4892	* @{

libavcodec/internal.h+13−0

@@ -160,6 +160,19 @@ typedef struct AVCodecInternal { · 159 unmodified lines
160	* hwaccel-specific private data	160	* hwaccel-specific private data
161	*/	161	*/
162	void *hwaccel_priv_data;	162	void *hwaccel_priv_data;
		163
		164	/**
		165	* checks API usage: after codec draining, flush is required to resume operation
		166	*/
		167	int draining;
		168
		169	/**
		170	* buffers for using new encode/decode API through legacy API
		171	*/
		172	AVPacket *buffer_pkt;
		173	int buffer_pkt_valid; // encoding: packet without data can be valid
		174	AVFrame *buffer_frame;
		175	int draining_done;
163	} AVCodecInternal;	176	} AVCodecInternal;
164		177
165	struct AVCodecDefault {	178	struct AVCodecDefault {

libavcodec/utils.c+287−2

@@ -167,12 +167,12 @@ static av_cold void avcodec_init(void) · 166 unmodified lines
167		167
168	int av_codec_is_encoder(const AVCodec *codec)	168	int av_codec_is_encoder(const AVCodec *codec)
169	{	169	{
170	return codec && (codec->encode_sub \|\| codec->encode2);	170	return codec && (codec->encode_sub \|\| codec->encode2 \|\|codec->send_frame);
171	}	171	}
172		172
173	int av_codec_is_decoder(const AVCodec *codec)	173	int av_codec_is_decoder(const AVCodec *codec)
174	{	174	{
175	return codec && codec->decode;	175	return codec && (codec->decode \|\| codec->send_packet);
176	}	176	}
177		177
178	av_cold void avcodec_register(AVCodec *codec)	178	av_cold void avcodec_register(AVCodec *codec)
@@ -1254,6 +1254,18 @@ int attribute_align_arg avcodec_open2(AVCodecContext avctx, const AVCodec code · 1075 unmodified lines
1254	goto free_and_end;	1254	goto free_and_end;
1255	}	1255	}
1256		1256
		1257	avctx->internal->buffer_frame = av_frame_alloc();
		1258	if (!avctx->internal->buffer_frame) {
		1259	ret = AVERROR(ENOMEM);
		1260	goto free_and_end;
		1261	}
		1262
		1263	avctx->internal->buffer_pkt = av_packet_alloc();
		1264	if (!avctx->internal->buffer_pkt) {
		1265	ret = AVERROR(ENOMEM);
		1266	goto free_and_end;
		1267	}
		1268
1257	if (codec->priv_data_size > 0) {	1269	if (codec->priv_data_size > 0) {
1258	if (!avctx->priv_data) {	1270	if (!avctx->priv_data) {
1259	avctx->priv_data = av_mallocz(codec->priv_data_size);	1271	avctx->priv_data = av_mallocz(codec->priv_data_size);
@@ -1765,6 +1777,11 @@ int attribute_align_arg avcodec_encode_audio2(AVCodecContext *avctx, · 505 unmodified lines
1765		1777
1766	*got_packet_ptr = 0;	1778	*got_packet_ptr = 0;
1767		1779
		1780	if (!avctx->codec->encode2) {
		1781	av_log(avctx, AV_LOG_ERROR, "This encoder requires using the avcodec_send_frame() API.\n");
		1782	return AVERROR(ENOSYS);
		1783	}
		1784
1768	if (!(avctx->codec->capabilities & AV_CODEC_CAP_DELAY) && !frame) {	1785	if (!(avctx->codec->capabilities & AV_CODEC_CAP_DELAY) && !frame) {
1769	av_packet_unref(avpkt);	1786	av_packet_unref(avpkt);
1770	av_init_packet(avpkt);	1787	av_init_packet(avpkt);
@@ -1904,6 +1921,11 @@ int attribute_align_arg avcodec_encode_video2(AVCodecContext *avctx, · 133 unmodified lines
1904		1921
1905	*got_packet_ptr = 0;	1922	*got_packet_ptr = 0;
1906		1923
		1924	if (!avctx->codec->encode2) {
		1925	av_log(avctx, AV_LOG_ERROR, "This encoder requires using the avcodec_send_frame() API.\n");
		1926	return AVERROR(ENOSYS);
		1927	}
		1928
1907	if(CONFIG_FRAME_THREAD_ENCODER &&	1929	if(CONFIG_FRAME_THREAD_ENCODER &&
1908	avctx->internal->frame_thread_encoder && (avctx->active_thread_type&FF_THREAD_FRAME))	1930	avctx->internal->frame_thread_encoder && (avctx->active_thread_type&FF_THREAD_FRAME))
1909	return ff_thread_video_encode_frame(avctx, avpkt, frame, got_packet_ptr);	1931	return ff_thread_video_encode_frame(avctx, avpkt, frame, got_packet_ptr);
@@ -2162,10 +2184,20 @@ int attribute_align_arg avcodec_decode_video2(AVCodecContext avctx, AVFrame pi · 252 unmodified lines
2162	return AVERROR(EINVAL);	2184	return AVERROR(EINVAL);
2163	}	2185	}
2164		2186
		2187	if (!avctx->codec->decode) {
		2188	av_log(avctx, AV_LOG_ERROR, "This decoder requires using the avcodec_send_packet() API.\n");
		2189	return AVERROR(ENOSYS);
		2190	}
		2191
2165	*got_picture_ptr = 0;	2192	*got_picture_ptr = 0;
2166	if ((avctx->coded_width \|\| avctx->coded_height) && av_image_check_size(avctx->coded_width, avctx->coded_height, 0, avctx))	2193	if ((avctx->coded_width \|\| avctx->coded_height) && av_image_check_size(avctx->coded_width, avctx->coded_height, 0, avctx))
2167	return AVERROR(EINVAL);	2194	return AVERROR(EINVAL);
2168		2195
		2196	avctx->internal->pkt = avpkt;
		2197	ret = apply_param_change(avctx, avpkt);
		2198	if (ret < 0)
		2199	return ret;
		2200
2169	av_frame_unref(picture);	2201	av_frame_unref(picture);
2170		2202
2171	if ((avctx->codec->capabilities & AV_CODEC_CAP_DELAY) \|\| avpkt->size \|\|	2203	if ((avctx->codec->capabilities & AV_CODEC_CAP_DELAY) \|\| avpkt->size \|\|
@@ -2247,6 +2279,11 @@ int attribute_align_arg avcodec_decode_audio4(AVCodecContext *avctx, · 75 unmodified lines
2247		2279
2248	*got_frame_ptr = 0;	2280	*got_frame_ptr = 0;
2249		2281
		2282	if (!avctx->codec->decode) {
		2283	av_log(avctx, AV_LOG_ERROR, "This decoder requires using the avcodec_send_packet() API.\n");
		2284	return AVERROR(ENOSYS);
		2285	}
		2286
2250	if (!avpkt->data && avpkt->size) {	2287	if (!avpkt->data && avpkt->size) {
2251	av_log(avctx, AV_LOG_ERROR, "invalid packet: NULL data, size != 0\n");	2288	av_log(avctx, AV_LOG_ERROR, "invalid packet: NULL data, size != 0\n");
2252	return AVERROR(EINVAL);	2289	return AVERROR(EINVAL);
@@ -2659,6 +2696,246 @@ void avsubtitle_free(AVSubtitle *sub) · 406 unmodified lines
2659	memset(sub, 0, sizeof(AVSubtitle));	2696	memset(sub, 0, sizeof(AVSubtitle));
2660	}	2697	}
2661		2698
		2699	static int do_decode(AVCodecContext avctx, AVPacket pkt)
		2700	{
		2701	int got_frame;
		2702	int ret;
		2703
		2704	av_assert0(!avctx->internal->buffer_frame->buf[0]);
		2705
		2706	if (!pkt)
		2707	pkt = avctx->internal->buffer_pkt;
		2708
		2709	// This is the lesser evil. The field is for compatibility with legacy users
		2710	// of the legacy API, and users using the new API should not be forced to
		2711	// even know about this field.
		2712	avctx->refcounted_frames = 1;
		2713
		2714	// Some codecs (at least wma lossless) will crash when feeding drain packets
		2715	// after EOF was signaled.
		2716	if (avctx->internal->draining_done)
		2717	return AVERROR_EOF;
		2718
		2719	if (avctx->codec_type == AVMEDIA_TYPE_VIDEO) {
		2720	ret = avcodec_decode_video2(avctx, avctx->internal->buffer_frame,
		2721	&got_frame, pkt);
		2722	if (ret >= 0)
		2723	ret = pkt->size;
		2724	} else if (avctx->codec_type == AVMEDIA_TYPE_AUDIO) {
		2725	ret = avcodec_decode_audio4(avctx, avctx->internal->buffer_frame,
		2726	&got_frame, pkt);
		2727	} else {
		2728	ret = AVERROR(EINVAL);
		2729	}
		2730
		2731	if (ret == AVERROR(EAGAIN))
		2732	ret = pkt->size;
		2733
		2734	if (ret < 0)
		2735	return ret;
		2736
		2737	if (avctx->internal->draining && !got_frame)
		2738	avctx->internal->draining_done = 1;
		2739
		2740	if (ret >= pkt->size) {
		2741	av_packet_unref(avctx->internal->buffer_pkt);
		2742	} else {
		2743	int consumed = ret;
		2744
		2745	if (pkt != avctx->internal->buffer_pkt) {
		2746	av_packet_unref(avctx->internal->buffer_pkt);
		2747	if ((ret = av_packet_ref(avctx->internal->buffer_pkt, pkt)) < 0)
		2748	return ret;
		2749	}
		2750
		2751	avctx->internal->buffer_pkt->data += consumed;
		2752	avctx->internal->buffer_pkt->size -= consumed;
		2753	avctx->internal->buffer_pkt->pts = AV_NOPTS_VALUE;
		2754	avctx->internal->buffer_pkt->dts = AV_NOPTS_VALUE;
		2755	}
		2756
		2757	if (got_frame)
		2758	av_assert0(avctx->internal->buffer_frame->buf[0]);
		2759
		2760	return 0;
		2761	}
		2762
		2763	int attribute_align_arg avcodec_send_packet(AVCodecContext avctx, const AVPacket avpkt)
		2764	{
		2765	int ret;
		2766
		2767	if (!avcodec_is_open(avctx) \|\| !av_codec_is_decoder(avctx->codec))
		2768	return AVERROR(EINVAL);
		2769
		2770	if (avctx->internal->draining)
		2771	return AVERROR_EOF;
		2772
		2773	if (!avpkt \|\| !avpkt->size) {
		2774	avctx->internal->draining = 1;
		2775	avpkt = NULL;
		2776
		2777	if (!(avctx->codec->capabilities & AV_CODEC_CAP_DELAY))
		2778	return 0;
		2779	}
		2780
		2781	if (avctx->codec->send_packet) {
		2782	if (avpkt) {
		2783	ret = apply_param_change(avctx, (AVPacket *)avpkt);
		2784	if (ret < 0)
		2785	return ret;
		2786	}
		2787	return avctx->codec->send_packet(avctx, avpkt);
		2788	}
		2789
		2790	// Emulation via old API. Assume avpkt is likely not refcounted, while
		2791	// decoder output is always refcounted, and avoid copying.
		2792
		2793	if (avctx->internal->buffer_pkt->size \|\| avctx->internal->buffer_frame->buf[0])
		2794	return AVERROR(EAGAIN);
		2795
		2796	// The goal is decoding the first frame of the packet without using memcpy,
		2797	// because the common case is having only 1 frame per packet (especially
		2798	// with video, but audio too). In other cases, it can't be avoided, unless
		2799	// the user is feeding refcounted packets.
		2800	return do_decode(avctx, (AVPacket *)avpkt);
		2801	}
		2802
		2803	int attribute_align_arg avcodec_receive_frame(AVCodecContext avctx, AVFrame frame)
		2804	{
		2805	int ret;
		2806
		2807	av_frame_unref(frame);
		2808
		2809	if (!avcodec_is_open(avctx) \|\| !av_codec_is_decoder(avctx->codec))
		2810	return AVERROR(EINVAL);
		2811
		2812	if (avctx->codec->receive_frame) {
		2813	if (avctx->internal->draining && !(avctx->codec->capabilities & AV_CODEC_CAP_DELAY))
		2814	return AVERROR_EOF;
		2815	return avctx->codec->receive_frame(avctx, frame);
		2816	}
		2817
		2818	// Emulation via old API.
		2819
		2820	if (!avctx->internal->buffer_frame->buf[0]) {
		2821	if (!avctx->internal->buffer_pkt->size && !avctx->internal->draining)
		2822	return AVERROR(EAGAIN);
		2823
		2824	while (1) {
		2825	if ((ret = do_decode(avctx, avctx->internal->buffer_pkt)) < 0) {
		2826	av_packet_unref(avctx->internal->buffer_pkt);
		2827	return ret;
		2828	}
		2829	// Some audio decoders may consume partial data without returning
		2830	// a frame (fate-wmapro-2ch). There is no way to make the caller
		2831	// call avcodec_receive_frame() again without returning a frame,
		2832	// so try to decode more in these cases.
		2833	if (avctx->internal->buffer_frame->buf[0] \|\|
		2834	!avctx->internal->buffer_pkt->size)
		2835	break;
		2836	}
		2837	}
		2838
		2839	if (!avctx->internal->buffer_frame->buf[0])
		2840	return avctx->internal->draining ? AVERROR_EOF : AVERROR(EAGAIN);
		2841
		2842	av_frame_move_ref(frame, avctx->internal->buffer_frame);
		2843	return 0;
		2844	}
		2845
		2846	static int do_encode(AVCodecContext avctx, const AVFrame frame, int *got_packet)
		2847	{
		2848	int ret;
		2849	*got_packet = 0;
		2850
		2851	av_packet_unref(avctx->internal->buffer_pkt);
		2852	avctx->internal->buffer_pkt_valid = 0;
		2853
		2854	if (avctx->codec_type == AVMEDIA_TYPE_VIDEO) {
		2855	ret = avcodec_encode_video2(avctx, avctx->internal->buffer_pkt,
		2856	frame, got_packet);
		2857	} else if (avctx->codec_type == AVMEDIA_TYPE_AUDIO) {
		2858	ret = avcodec_encode_audio2(avctx, avctx->internal->buffer_pkt,
		2859	frame, got_packet);
		2860	} else {
		2861	ret = AVERROR(EINVAL);
		2862	}
		2863
		2864	if (ret >= 0 && *got_packet) {
		2865	// Encoders must always return ref-counted buffers.
		2866	// Side-data only packets have no data and can be not ref-counted.
		2867	av_assert0(!avctx->internal->buffer_pkt->data \|\| avctx->internal->buffer_pkt->buf);
		2868	avctx->internal->buffer_pkt_valid = 1;
		2869	ret = 0;
		2870	} else {
		2871	av_packet_unref(avctx->internal->buffer_pkt);
		2872	}
		2873
		2874	return ret;
		2875	}
		2876
		2877	int attribute_align_arg avcodec_send_frame(AVCodecContext avctx, const AVFrame frame)
		2878	{
		2879	if (!avcodec_is_open(avctx) \|\| !av_codec_is_encoder(avctx->codec))
		2880	return AVERROR(EINVAL);
		2881
		2882	if (avctx->internal->draining)
		2883	return AVERROR_EOF;
		2884
		2885	if (!frame) {
		2886	avctx->internal->draining = 1;
		2887
		2888	if (!(avctx->codec->capabilities & AV_CODEC_CAP_DELAY))
		2889	return 0;
		2890	}
		2891
		2892	if (avctx->codec->send_frame)
		2893	return avctx->codec->send_frame(avctx, frame);
		2894
		2895	// Emulation via old API. Do it here instead of avcodec_receive_packet, because:
		2896	// 1. if the AVFrame is not refcounted, the copying will be much more
		2897	// expensive than copying the packet data
		2898	// 2. assume few users use non-refcounted AVPackets, so usually no copy is
		2899	// needed
		2900
		2901	if (avctx->internal->buffer_pkt_valid)
		2902	return AVERROR(EAGAIN);
		2903
		2904	return do_encode(avctx, frame, &(int){0});
		2905	}
		2906
		2907	int attribute_align_arg avcodec_receive_packet(AVCodecContext avctx, AVPacket avpkt)
		2908	{
		2909	av_packet_unref(avpkt);
		2910
		2911	if (!avcodec_is_open(avctx) \|\| !av_codec_is_encoder(avctx->codec))
		2912	return AVERROR(EINVAL);
		2913
		2914	if (avctx->codec->receive_packet) {
		2915	if (avctx->internal->draining && !(avctx->codec->capabilities & AV_CODEC_CAP_DELAY))
		2916	return AVERROR_EOF;
		2917	return avctx->codec->receive_packet(avctx, avpkt);
		2918	}
		2919
		2920	// Emulation via old API.
		2921
		2922	if (!avctx->internal->buffer_pkt_valid) {
		2923	int got_packet;
		2924	int ret;
		2925	if (!avctx->internal->draining)
		2926	return AVERROR(EAGAIN);
		2927	ret = do_encode(avctx, NULL, &got_packet);
		2928	if (ret < 0)
		2929	return ret;
		2930	if (ret >= 0 && !got_packet)
		2931	return AVERROR_EOF;
		2932	}
		2933
		2934	av_packet_move_ref(avpkt, avctx->internal->buffer_pkt);
		2935	avctx->internal->buffer_pkt_valid = 0;
		2936	return 0;
		2937	}
		2938
2662	av_cold int avcodec_close(AVCodecContext *avctx)	2939	av_cold int avcodec_close(AVCodecContext *avctx)
2663	{	2940	{
2664	int i;	2941	int i;
@@ -2679,6 +2956,8 @@ av_cold int avcodec_close(AVCodecContext *avctx) · 14 unmodified lines
2679	avctx->internal->byte_buffer_size = 0;	2956	avctx->internal->byte_buffer_size = 0;
2680	av_freep(&avctx->internal->byte_buffer);	2957	av_freep(&avctx->internal->byte_buffer);
2681	av_frame_free(&avctx->internal->to_free);	2958	av_frame_free(&avctx->internal->to_free);
		2959	av_frame_free(&avctx->internal->buffer_frame);
		2960	av_packet_free(&avctx->internal->buffer_pkt);
2682	for (i = 0; i < FF_ARRAY_ELEMS(pool->pools); i++)	2961	for (i = 0; i < FF_ARRAY_ELEMS(pool->pools); i++)
2683	av_buffer_pool_uninit(&pool->pools[i]);	2962	av_buffer_pool_uninit(&pool->pools[i]);
2684	av_freep(&avctx->internal->pool);	2963	av_freep(&avctx->internal->pool);
@@ -3049,6 +3328,12 @@ const char *avcodec_license(void) · 364 unmodified lines
3049		3328
3050	void avcodec_flush_buffers(AVCodecContext *avctx)	3329	void avcodec_flush_buffers(AVCodecContext *avctx)
3051	{	3330	{
		3331	avctx->internal->draining = 0;
		3332	avctx->internal->draining_done = 0;
		3333	av_frame_unref(avctx->internal->buffer_frame);
		3334	av_packet_unref(avctx->internal->buffer_pkt);
		3335	avctx->internal->buffer_pkt_valid = 0;
		3336
3052	if (HAVE_THREADS && avctx->active_thread_type & FF_THREAD_FRAME)	3337	if (HAVE_THREADS && avctx->active_thread_type & FF_THREAD_FRAME)
3053	ff_thread_flush(avctx);	3338	ff_thread_flush(avctx);
3054	else if (avctx->codec->flush)	3339	else if (avctx->codec->flush)

libavcodec/version.h+1−1

@@ -28,7 +28,7 @@ · 27 unmodified lines
28	#include "libavutil/version.h"	28	#include "libavutil/version.h"
29		29
30	#define LIBAVCODEC_VERSION_MAJOR 57	30	#define LIBAVCODEC_VERSION_MAJOR 57
31	#define LIBAVCODEC_VERSION_MINOR 36	31	#define LIBAVCODEC_VERSION_MINOR 37
32	#define LIBAVCODEC_VERSION_MICRO 100	32	#define LIBAVCODEC_VERSION_MICRO 100
33		33
34	#define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \	34	#define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \

libavformat/avformat.h+2−2

@@ -161,8 +161,8 @@ · 160 unmodified lines
161	* av_read_frame() on it. Each call, if successful, will return an AVPacket	161	* av_read_frame() on it. Each call, if successful, will return an AVPacket
162	* containing encoded data for one AVStream, identified by	162	* containing encoded data for one AVStream, identified by
163	* AVPacket.stream_index. This packet may be passed straight into the libavcodec	163	* AVPacket.stream_index. This packet may be passed straight into the libavcodec
164	* decoding functions avcodec_decode_video2(), avcodec_decode_audio4() or	164	* decoding functions avcodec_send_packet() or avcodec_decode_subtitle2() if the
165	* avcodec_decode_subtitle2() if the caller wishes to decode the data.	165	* caller wishes to decode the data.
166	*	166	*
167	* AVPacket.pts, AVPacket.dts and AVPacket.duration timing information will be	167	* AVPacket.pts, AVPacket.dts and AVPacket.duration timing information will be
168	* set if known. They may also be unset (i.e. AV_NOPTS_VALUE for	168	* set if known. They may also be unset (i.e. AV_NOPTS_VALUE for

lavc: introduce a new decoding/encoding API with decoupled input/output

Keyboard shortcuts

General

Issues and pull requests

Issues, pull requests and Inbox

Repository code