Play a mp4 video on embedded linux

Questions and information about Crank Software's Storyboard Engine

Moderator: jamiev

Re: Play a mp4 video on embedded linux

Postby rdowdall » Mon Nov 12, 2018 9:30 am

Hello Damon

You want your gstreamer-backend command line to be:

/opt/storyboard/bin/gstreamer-backend -v -p "filesrc location=/mnt/media/movies/Vario_Brush_mpeg4_1024x576_15fps_QP7_ok.mp4 ! qtdemux ! ffdec_mpeg4 ! ffmpegcolorspace ! fbdevsink device=/dev/fb1" > gstreamer-out.txt 2>&1

Run this before you run your Storyboard application. Then launch your Storyboard application with the same command line arguments that you have been using, but add in -vvvv before path to your gapp file and redirect that output to a file named sbengine-out.txt, so like this:

sbengine -vvvv <your_options> <path_to_gapp_file/gapp_file> > sbengine-out.txt 2>&1

Then click on the play button in your Storyboard application. Make sure that the ffmpeg-plugin is not in your plugins directory on the target.

With the setup that you have, the video is going to play over top of the Storyboard application.

Thanks,
Rodney
rdowdall
 
Posts: 124
Joined: Mon Nov 02, 2009 9:46 pm

Re: Play a mp4 video on embedded linux

Postby damon.feng » Wed Nov 14, 2018 2:47 am

Hello Rdowdall,

I added the following command before launching my application.
opt/storyboard/bin/gstreamer-backend -p "filesrc location=/mnt/media/movies/Vario_Brush_mpeg4_1024x576_15fps_QP7_ok.mp4 ! qtdemux ! ffdec_mpeg4 ! ffmpegcolorspace ! fbdevsink device=/dev/fb1" &
The video can play back eventually. However, I cannot choose any other video to play, although I set some necessary parameters to my application with variables including media_name, output_width, and output_height just like the below screenshot.
Capture8.JPG
Capture8.JPG (143.9 KiB) Viewed 128 times


In addition, I met another issue. I cannot pause the video with an action [gra.media.playpause]. The debug log looks like this.
2018-11-14T14:49:36.261637 [info] UI_Touchscreen_GUI: WARN [137.922]:ACTION: Missing action handler [gra.media.playpause] ... missing a plugin?
I don't know which plugin is missing.
damon.feng
 
Posts: 8
Joined: Wed Oct 31, 2018 1:20 am

Re: Play a mp4 video on embedded linux

Postby rdowdall » Wed Nov 14, 2018 10:28 am

Hello Damon

I made a mistake in the pipeline that I sent you. You need to specify it as:

opt/storyboard/bin/gstreamer-backend -p "filesrc location=/mnt/media/movies/Vario_Brush_mpeg4_1024x576_15fps_QP7_ok.mp4 name=media-src ! qtdemux ! ffdec_mpeg4 ! ffmpegcolorspace ! fbdevsink device=/dev/fb1" &

Without the name option, the gstreamer-backend application can't find the file source element in the pipeline, which means that it can't change the video name.

As for the playpause actions, it looks like you are using an old media.sbat file as those actions have been renamed to gra.media.pause and gra.media.resume. Attached is an updated media.sbat file. It has been uploaded with the extension .txt, so after downloading it rename it to media.sbat. Replace the one that you have in the templates directory of your project with the one attached. Then close your application and open it again.

Where did you get your media.sbat file from? Was it from the sample? I only ask because it looks like we need to update it at the location from where you got it.

Thanks,
Rodney
Attachments
media.txt
Current media.sbat file. Rename to media.sbat when downloaded and copy to the projects "template" directory in Designer. Then close and reopen the project.
(2.74 KiB) Downloaded 7 times
rdowdall
 
Posts: 124
Joined: Mon Nov 02, 2009 9:46 pm

Re: Play a mp4 video on embedded linux

Postby damon.feng » Wed Nov 14, 2018 9:58 pm

Hello Rdowdall,

I can choose any video to play now as you instructed now.

But as for the playpause actions, I still failed. I substitute the media.sbat as you said. But I still cannot find the action "gra.media.pause" from the action list, and all my self-defined events disappeared.
Capture9.JPG
Capture9.JPG (65.32 KiB) Viewed 117 times
damon.feng
 
Posts: 8
Joined: Wed Oct 31, 2018 1:20 am

Re: Play a mp4 video on embedded linux

Postby rdowdall » Wed Nov 14, 2018 10:33 pm

Hello Damon

Something appears to be wrong with the file that I posted, but I can't spot where. Can you please paste this:

<actiontemplates>
<template name="gra.media.new.audio">
<arguments>
<element name="channel_name" type="string" />
<element name="media_name" type="string" />
<element name="volume" type="integer" />
<element name="update_interval" type="integer" />
<element name="emit_time_events" type="integer" />
<element name="extra_data" type="string" />
</arguments>
</template>
<template name="gra.media.new.video">
<arguments>
<element name="channel_name" type="string" />
<element name="media_name" type="string" />
<element name="volume" type="integer" />
<element name="update_interval" type="integer" />
<element name="emit_time_events" type="integer" />
<element name="external_buffer_name" type="string" />
<element name="object_name" type="string" />
<element name="output_width" type="integer" />
<element name="output_height" type="integer" />
<element name="output_depth" type="integer" />
<element name="extra_data" type="string" />
</arguments>
</template>
<template name="gra.media.volume">
<arguments>
<element name="channel_name" type="string" />
<element name="volume" type="integer" />
<element name="emit_volume_event" type="integer" />
</arguments>
</template>
<template name="gra.media.seek">
<arguments>
<element name="channel_name" type="string" />
<element name="seek_num" type="integer" />
<element name="emit_state_event" type="integer" />
</arguments>
</template>
<template name="gra.media.stop">
<arguments>
<element name="channel_name" type="string" />
<element name="emit_state_event" type="integer" />
</arguments>
</template>
<template name="gra.media.pause">
<arguments>
<element name="channel_name" type="string" />
<element name="emit_state_event" type="integer" />
</arguments>
</template>
<template name="gra.media.resume">
<arguments>
<element name="channel_name" type="string" />
<element name="emit_state_event" type="integer" />
</arguments>
</template>
<template name="gra.media.connect">
</template>
<template name="gra.media.disconnect">
</template>
</actiontemplates>

In to your media.sbat file and reload the project.

The missing events problem is weird. The custom events are stored in the file events/custom.evt in the project. Did that file get deleted by accident?

Thanks,
Rodney
rdowdall
 
Posts: 124
Joined: Mon Nov 02, 2009 9:46 pm

Re: Play a mp4 video on embedded linux

Postby damon.feng » Fri Nov 16, 2018 3:12 am

Hello, Rdowdall,

Thank you for your help! Now I've already been able to play and pause videos on my target board.

However, I still have one issue about grabbing the last frame of a video. If you can continue giving a hand, I will be very grateful.

When I was trying to grab the last frame of a video, I used the following commands.
cp /dev/fb1 /tmp/screenshot.raw
ffmpeg -f rawvideo -pix_fmt rgb32 -s 1280x800 -i /tmp/screenshot.raw -y /tmp/lastframe%01d.jpg

But I met one error.
ffmpeg version 3.2.3 Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 5.4.0 (Buildroot 2017.02.5-g97252bd)
configuration: --enable-cross-compile --cross-prefix=/jenkins/workspace/Sentinel/Sentinel-git/buildroot-2017.02.5/output/host/usr/bin/arm-buildroot-linux-gnueabihf- --sysroot=/jenkins/workspace/S/output/host/usr/arm-buildroot-linux-gnueabihf/sysroot --host-cc=/usr/bin/gcc --arch=arm --target-os=linux --disable-stripping --pkg-config=/jenkins/workspace/Sentinel/Sentinel-git/buildroot-2017.0able-static --enable-shared --prefix=/usr --enable-avfilter --disable-version3 --enable-logging --enable-optimizations --disable-extra-warnings --enable-avdevice --enable-avcodec --enable-avformat able-gray --enable-swscale-alpha --disable-small --enable-dct --enable-fft --enable-mdct --enable-rdft --disable-crystalhd --disable-dxva2 --enable-runtime-cpudetect --disable-hardcoded-tables --didisable-mipsdspr2 --disable-msa --enable-hwaccels --disable-cuda --disable-cuvid --disable-nvenc --disable-avisynth --disable-frei0r --disable-libopencore-amrnb --disable-libopencore-amrwb --disabllibgsm --disable-libilbc --disable-libnut --disable-libopenjpeg --disable-libschroedinger --disable-libvo-amrwbenc --disable-symver --disable-doc --disable-gpl --disable-nonfree --enable-ffmpeg --ee-avresample --disable-ffprobe --disable-postproc --enable-swscale --enable-indevs --enable-outdevs --enable-pthreads --enable-zlib --disable-bzlib --disable-libfdk-aac --disable-gnutls --enable-opvorbis --enable-muxer=ogg --enable-encoder=libvorbis --disable-vaapi --disable-vdpau --disable-libopencv --disable-libopus --disable-libvpx --disable-libass --disable-libbluray --disable-librtmp ----disable-libspeex --disable-libtheora --disable-libwavpack --enable-libfreetype --enable-fontconfig --disable-libx264 --disable-libx265 --disable-yasm --disable-mmx --disable-sse --disable-sse2 --se4 --disable-sse42 --disable-avx --disable-avx2 --enable-armv6 --enable-vfp --enable-neon --disable-altivec --enable-pic --cpu=cortex-a9
libavutil 55. 34.101 / 55. 34.101
libavcodec 57. 64.101 / 57. 64.101
libavformat 57. 56.101 / 57. 56.101
libavdevice 57. 1.100 / 57. 1.100
libavfilter 6. 65.100 / 6. 65.100
libswscale 4. 2.100 / 4. 2.100
libswresample 2. 3.100 / 2. 3.100
[rawvideo @ 0x63fc0] Estimating duration from bitrate, this may be inaccurate
Input #0, rawvideo, from '/tmp/screenshot.raw':
Duration: 00:00:00.04, start: 0.000000, bitrate: 838860 kb/s
Stream #0:0: Video: rawvideo (RGBA / 0x41424752), rgba, 1280x800, 819200 kb/s, 25 tbr, 25 tbn, 25 tbc
[swscaler @ 0x74b80] deprecated pixel format used, make sure you did set range correctly
Output #0, image2, to '/mnt/media/lastframe%01d.jpg':
Metadata:
encoder : Lavf57.56.101
Stream #0:0: Video: mjpeg, yuvj444p(pc), 1280x800, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
Metadata:
encoder : Lavc57.64.101 mjpeg
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> mjpeg (native))
Press [q] to stop, [?] for help
[rawvideo @ 0x6d6d0] Invalid buffer size, packet size 98304 < expected frame_size 4096000
Error while decoding stream #0:0: Invalid argument

frame= 1 fps=0.0 q=4.9 Lsize=N/A time=00:00:00.04 bitrate=N/A speed=0.0529x
video:88kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
damon.feng
 
Posts: 8
Joined: Wed Oct 31, 2018 1:20 am

Re: Play a mp4 video on embedded linux

Postby rdowdall » Fri Nov 16, 2018 9:45 am

Hello Damon

I'm not exactly sure what the error is there. Is your framebuffer actually 1280x800? What does fbset say? It could be that the resolution is something else so it's not reading in as many pixels as it thinks it should be.

You could try playing the screen grab with ffplay to see if it can recognize it:

ffplay -f rawvideo -pixel_format rgb24 -video_size 1280x800 -framerate 1 /tmp/screenshot.raw

It looks like it is trying to save the jpeg in yuv format based on this output:

Stream #0:0: Video: mjpeg, yuvj444p(pc), 1280x800, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc

and maybe that conversion is failing.

FFmpeg might be heavy-handed in this case though if all you are looking for is a jpeg of the last video frame. You can still do your output to the raw file, and then you could use https://www.imagemagick.org/script/index.php to convert it to a jpeg. Just save the file as .rgb instead of .raw and run the convert command on it. Something like:

convert -size "1280x800" -depth 32 /tmp/screenshot.rgb /tmp/lastframe.jpg

Barring that, you could write a simple application that reads the raw file and then uses libjpeg to output the jpeg file.

Thanks,
Rodney
rdowdall
 
Posts: 124
Joined: Mon Nov 02, 2009 9:46 pm

Previous

Return to Storyboard Engine

Who is online

Users browsing this forum: No registered users and 1 guest

cron