MediaFoundation: TimeStamp of first frame is not getting correctly set to 0. Instead, it is getting set as 0.0333 (frame duration)
I'm trying to read two frames, each individually stored in binary files named frame1.bin and frame2.bin, and create an MP4 video file called sample.mp4.
Here's what I'm doing:
I read a frame, associate it with its respective timestamp, and write it to the video file.
CComPtr<IMFMediaBuffer> mediaBuffer(createMediaBuffer(input));
// Create a media sample and add the buffer to the sample.
CComPtr<IMFSample> sample;
MFWRITER_RETURN_ON_ERROR(throwIf(MFCreateSample(&sample)));
MFWRITER_RETURN_ON_ERROR(throwIf(sample->AddBuffer(mediaBuffer)));
// Set the time stamp and the duration.
MFWRITER_RETURN_ON_ERROR(throwIf(sample->SetSampleTime(_frameStart)));
MFWRITER_RETURN_ON_ERROR(throwIf(sample->SetSampleDuration(_frameDuration)));
// Send the sample to the Sink Writer.
MFWRITER_RETURN_ON_ERROR(throwIf(_sinkWriter->WriteSample(_videoStreamIndex, sample)));
// Advance the starting position for the next frame
_frameStart += _frameDuration;
I create and set the properties of sinkWriter as below:
std::wstring wideFile = L"sample.mp4";
CComPtr<IMFMediaType> pMediaTypeOut, pMediaTypeIn;
MFWRITER_RETURN_HRESULT_ON_ERROR(MFCreateSinkWriterFromURL(wideFile.c_str(), NULL, NULL, &_sinkWriter));
// Set the output media type.
MFWRITER_RETURN_HRESULT_ON_ERROR(MFCreateMediaType(&pMediaTypeOut));
MFWRITER_RETURN_HRESULT_ON_ERROR(pMediaTypeOut->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video));
MFWRITER_RETURN_HRESULT_ON_ERROR(pMediaTypeOut->SetGUID(MF_MT_SUBTYPE, MFVideoFormat_H264));
MFWRITER_RETURN_HRESULT_ON_ERROR(pMediaTypeOut->SetUINT32(MF_MT_MPEG2_PROFILE, eAVEncH264VProfile_Main));
// Set the Bit rate. NOTE this assumes the input data is natively 24 bits per pixel (8 per band, 3 bands)
MFWRITER_RETURN_HRESULT_ON_ERROR(pMediaTypeOut->SetUINT32(MF_MT_AVG_BITRATE, computeH264BitRate(_quality, width, height, _frameRate, 24)));
MFWRITER_RETURN_HRESULT_ON_ERROR(pMediaTypeOut->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlace_Progressive));
MFWRITER_RETURN_HRESULT_ON_ERROR(MFSetAttributeSize(pMediaTypeOut, MF_MT_FRAME_SIZE, width, height));
_frameDuration = static_cast<long long>(std::round((1.0 / _frameRate) * HUNDRED_NANOSECONDS));
// Derive the frame rate from
UINT32 rateNumerator = 0, rateDenominator = 0;
MFWRITER_RETURN_HRESULT_ON_ERROR(MFAverageTimePerFrameToFrameRate(_frameDuration, &rateNumerator, &rateDenominator));
MFWRITER_RETURN_HRESULT_ON_ERROR(MFSetAttributeRatio(pMediaTypeOut, MF_MT_FRAME_RATE, rateNumerator, rateDenominator));
MFWRITER_RETURN_HRESULT_ON_ERROR(MFSetAttributeRatio(pMediaTypeOut, MF_MT_PIXEL_ASPECT_RATIO, 1, 1));
MFWRITER_RETURN_HRESULT_ON_ERROR(_sinkWriter->AddStream(pMediaTypeOut, &_videoStreamIndex));
// Set the input media type.
MFWRITER_RETURN_HRESULT_ON_ERROR(MFCreateMediaType(&pMediaTypeIn));
MFWRITER_RETURN_HRESULT_ON_ERROR(pMediaTypeIn->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video));
MFWRITER_RETURN_HRESULT_ON_ERROR(pMediaTypeIn->SetGUID(MF_MT_SUBTYPE, MFVideoFormat_YUY2));
MFWRITER_RETURN_HRESULT_ON_ERROR(pMediaTypeIn->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlace_Progressive));
MFWRITER_RETURN_HRESULT_ON_ERROR(MFSetAttributeSize(pMediaTypeIn, MF_MT_FRAME_SIZE, width, height));
MFWRITER_RETURN_HRESULT_ON_ERROR(MFSetAttributeRatio(pMediaTypeIn, MF_MT_FRAME_RATE, rateNumerator, rateDenominator));
MFWRITER_RETURN_HRESULT_ON_ERROR(MFSetAttributeRatio(pMediaTypeIn, MF_MT_PIXEL_ASPECT_RATIO, 1, 1));
MFWRITER_RETURN_HRESULT_ON_ERROR(_sinkWriter->SetInputMediaType(_videoStreamIndex, pMediaTypeIn, NULL));
_frameStart = 0;
// Tell the sink writer to start accepting data.
MFWRITER_RETURN_HRESULT_ON_ERROR(_sinkWriter->BeginWriting());
Issue I am facing:
The timestamp is starting at 0.0333 instead of 0, which seems to correspond to the frame duration value. I printed the _frameStart value before and after associating it with the sample using SetSampleTime, and it printed 0 in both cases, as expected. However, in the generated MP4 file, the timestamp of the first frame is 0.0333 instead of 0. I am unable to determine the cause of this issue.