[Mp4-tech][video][H.263] Relative output order of B-pictures and B-parts of PB-pictures?

[J. Miles]

Hi Gary,
Thank you for responding, and I am sorry for posting it to the MPEG-4 list.
I am also working on with the MPEG-4 standards and have posted other
questions to this list, so I was simply not thinking clearly when I posted
it here. I was not aware of the other mail list - thank you for making me
aware of that. I have kept the MPEG-4 list in the loop on this reply since I
have know started the discussion the replies may still be of interest to
others.
You are right; I do also find it somewhat weird to combine PB- and
B-pictures, and I am not really expecting any actual use of this
combination. Nevertheless, I was trying to make the decoder as complete as
possible and hence I ran into the issue with this combination. I do agree
that a mode restriction disallowing simultaneous use of B-pictures and
PB-pictures would have been beneficial. It seems then, however, that a
simple restriction is not possible, at-least not without also changing the
text in Annex O, because PB-pictures are mentioned in that annex. Therefore,
it seems to me that the simultaneous use of PB-pictures has been a part of
the design of the standard (see e.g. section O.1.4). Anyway, this is
probably a moot point as there is little interest in changing or clarifying
the standard, which I fully appreciate as everyone now seems to be going
with H.264.
I find it interesting that you think that there is no (or very limited)
actual use of PB-pictures. You think that also applies to B-pictures? With
that being the case, you have made me wondering, why these features were
at-all put into the standard - especially considering that Improved
PB-pictures, and to some extent also B-pictures, are included later in the
extensions of the standard. If there was an interest/demand for these
features then - why no practical usage?
I do agree that the combination [1] with the B-part preceding the B-pictures
in output order would be best in practice to reduce the required amount of
buffering. I also ran into that consideration. But if it is not defined to
be so in the standard, the decoder cannot in general assume anything and
will thus need to be prepared for the additional buffer. Only through
(unspecified) external means can one control the amount of buffering
required. Thanks for clarifying that the output ordering would still follow
the TR and TRB syntax elements. I do think that temporal reference and
output ordering in general could have been more clearly described in the
standard, though.
Thanks again.
Best regards,
Jay
On Tue, May 20, 2024 at 7:37 PM, Gary Sullivan <
garysull windows.microsoft.com> wrote:
>  Jay Miles et al,
>
> I've never seen that particular question come up before.  The MPEG-4
> technology reflector is really not the right place to ask the question,
> though, because those modes of H.263 are not found in MPEG-4 video.  The
> right place to ask is vceg-experts yahoogroups.com, so I'm cross-posting
> to that list.
>
> That is really a rather obscure question, as I doubt anyone would actually
> use those two features together.  (Even finding implementations that use
> those features individually is probably difficult.)  But the standard does
> say that it's possible, so (in principle) there ought to be an answer to the
> question.  In hindsight, it might have been a better idea to just disallow
> that combination.
>
>  Hypothetically, I think there could be other conceivable orders as well
> as the two that you listed, such as having B2 fall temporally between B3 and
> B4 for output.
>
> I think the encoder's intent would be clearly expressed by the TR syntax
> element of the B frame and the TRB syntax element of the PB (or Improved PB)
> frame.
>
> However, I think only output order 1 (where the B part of the PB2 frame
> temporally precedes the true-B pictures for output) would actually be
> reasonable from an implementation requirement perspective.  Output order 2
> would require an extra frame store in a decoder, as the decoder would need
> to store the decoded B part of the PB2 frame until after decoding and output
> of the B3 and P4 pictures, whereas the other order would not have that
> requirement.
>
> Test bitstreams for H.263 are hard to find in general (e.g., there are no
> official test bitstreams).  An obscure one like that is probably impossible
> to find.
>
> Probably today both of those optional features of H.263 operation are
> mostly only of academic interest as historical reference designs.  That
> would especially seem to be true of the PB frames (and Improved PB frames)
> feature.
>
> Hypothetically, we could create an "implementer's guide" for H.263 to work
> toward clarification of that aspect.  I'm not aware of any other open
> questions regarding errata issues in H.263.
>
> Best Regards,
>
> Gary Sullivan
>
>
>  ------------------------------
> *From:* mp4-tech-bounces lists.mpegif.org [mailto:
> mp4-tech-bounces lists.mpegif.org] *On Behalf Of *J. Miles
> *Sent:* Tuesday, May 20, 2024 2:00 AM
> *To:* mp4-tech lists.mpegif.org
> *Subject:* [Mp4-tech][video][H.263] Relative output order of B-pictures
> and B-parts of PB-pictures?
>
> Hi everyone.
>
> I am working with H.263 decoding, and I hope someone can help me clarifying
> the relative output order of B-pictures and B-parts of PB-pictures (or
> Improved PB-pictures) in H.263. I do not find the standard clear on these
> subjects. From the standard it is clear that the B-part of a PB-picture is
> always output before the P-part, which is also intuitive to the usual
> definition of B-pictures from e.g. MPEG-2 (=H.262) for enabling
> bi-predicition. The same seems to go for B-pictures when just having these
> in the stream (i.e. no PB-pictures). For example, a simple stream with just
> two PB-pictures between each P-picture will be output as follows:
>
>  Coded input order:    I0 | P1 | PB2 | PB3 | P4 | PB5 | . . .
>
>  Decoded output order: I0 | P1 | B2* | P2 | B3* | P3 | P4 | B5* | P5 |. . .
>
> How would the relative output order look for a stream containing both
> 'true' B-pictures and B-parts of PB-pictures? Would the B-parts come out
> first (similar to a stream without B-pictures as in the example above), or
> are the B-pictures output first? In the example below there are two
> B-pictures between each PB-picture, and the first output ordering in [1]
> outputs the B-parts first (marked with *), whereas in [2] the B-pictures are
> output first.
>
>  Coded input order:        I0 | P1 | PB2 | B3 | B4 | PB5 | B6 | B7 | . . .
>
>  Decoded output order [1]: I0 | P0 | B2* | B3 | B4 | P2| B5* | B6 | B7 | P5
> | . . .
>
>  Decoded output order [2]: I0 | P0 | B3 | B4 | B2* | P2| B6 | B7 | B5* | P5
> | . . .
>
> What is the correct handling of output ordering?
>
> I am also wondering if anyone know of any test streams that contain both
> B-pictures and PB-pictures that I can use for testing/checking?
>
> Any help and comments would be appreciated. Thanks.
>
> Jay
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: /pipermail/mp4-tech/attachments/20080526/4bd1d37f/attachment-0001.html