[Mp4-tech] [H.264] purpose of non-existing pictures

[Wesley De Neve]

Dear Martin,
I think you rather have to look at "non-existing" frames from a decoder 
point of view. Then, temporal scalability is a possible scenario in which 
gaps might be introduced in frame_num. Frame loss due to packet dropping in 
a congested network is another one.
When no gaps in frame_num are allowed (the value of 
gaps_in_frame_num_value_allowed_flag is equal to zero), decoders infer 
disposed reference frames from gaps in the frame_num syntax element and 
insert a "non-existing'' frame to the reference picture buffer as if the 
frame was received and decoded. This procedure guarantees that the contents 
of and the picture order in the reference picture buffer remain unchanged 
when it comes to the decoding of the remaining pictures. This process is not 
executed when gaps in frame_num are allowed (the value of 
gaps_in_frame_num_value_allowed_flag is equal to one). The latter is for 
instance useful when temporal scalability is implemented by making use of 
sub-sequences. The sub-sequence feature of H.264/AVC enables hierarchical 
temporal scalability, which allows disposal of reference pictures from a 
coded bitstream without affecting the decoding of the remaining pictures in 
the bitstream (hereby intentionally introducing gaps in frame_num when 
reference pictures are dropped).
The latter is also discussed in the following papers:
[1] Dong Tian et al.: Sub-Sequence Video Coding for Improved Temporal 
Scalability;
[2] Ville-Pekka Limnell et al.: Quality Scalability in H.264-AVC Video 
Coding.
>From an encoder perspective, "non-existing" frames will typically be created 
by using skipped slices in order to achieve bit rate savings (or to keep the 
amount of pictures in the coded bitstream the same after offline 
exploitation of temporal scalability, something that significantly eases the 
synchronization with an audio stream when varying coding patterns are in 
use).
Best regards,
Wesley De Neve
Martin.Lange sci-worx.com wrote:
> Hello everyone,
>
> a thing which I was always wondering is the purpose of non-existing
> pictures. What is a scenario in which an encoder would decide to
> intentionally introduce gaps in frame_num, thus producing non-existing
> pictures? The only effect I see is that they make other pictures fall
> out of the decoded picture buffer and that they also get incorporated
> into reference lists, thus changing the reference indices other
> pictures
> would have without their presence. Can this lead to any gain in coding
> efficiency? Can anybody comment on what was the intention of this
> tool?
>
> If this is a question which has already been posted in the past, I
> would
> be happy if anyone could point me to some kind of FAQ of this forum,
> if
> it exists (does it?).
>
> Best regards,
> Martin Lange
>
> _______________________________________________
> NOTE: Please use clear subject lines for your posts. Include [audio,
> [video], [systems], [general] or another apppropriate identifier to
> indicate the type of question you have.
>
> Note: Conduct on the mailing list is subject to the Antitrust
> guidelines found at
> http://www.mpegif.org/public/documents/vault/mp-out-30042-Antitrust.php