dew.data.sources.av_utils
Frames and sound of a media file over a time window, which video.PickClip draws once for both.
moviepy is the reader because it is what the av extra installs and it
reads from wheels alone; it drives ffmpeg, so nothing here holds a decoder
open across records.
| Name | Summary |
|---|---|
FPS | The frame rate a clip is read at where none is asked for (video.DecodeVideo). |
media_info | path’s duration in seconds and whether it holds a video and an audio stream, from ffmpeg’s reading of the container, so a sound file answers too. |
audio_window | The sound of path from start for duration seconds, mono at sample_rate, float32 in [-1, 1], and whether the track holds each sample: silence and False before the track begins, whether start is before the file’s or the track starts late, and after it ends. |
read_frames | count frames of path at start + i / fps seconds, as uint8 [count, H, W, 3] RGB. |
FPS = 25.0The frame rate a clip is read at where none is asked for (video.DecodeVideo).
media_info
Section titled “media_info”def media_info(path: str) -> tuple[float, bool, bool]path’s duration in seconds and whether it holds a video and an audio stream, from ffmpeg’s
reading of the container, so a sound file answers too.
audio_window
Section titled “audio_window”def audio_window( path: str, start: float, duration: float, sample_rate: int,) -> tuple[np.ndarray, np.ndarray]The sound of path from start for duration seconds, mono at sample_rate, float32 in
[-1, 1], and whether the track holds each sample: silence and False before the track begins,
whether start is before the file’s or the track starts late, and after it ends.
ffmpeg does the seek, the resampling and the downmix, through the binary moviepy drives. moviepy’s own audio reader asks for stereo at 44.1 kHz whatever the track is, and ffmpeg’s mono-to-stereo upmix takes 3 dB off a mono track; sampling that stream at another rate then picks the nearest 44.1 kHz sample, which aliases and is never sample-exact.
The samples are placed by the timestamp of the first one decoded, read a
second before start: cut by ffmpeg’s own seek instead, an AVI’s track
started 73 or 261 samples late in 2 of 80 windows and a Matroska one up
to 464 samples off, and a track that starts after the video came back
shifted to the window’s start. Placed, all 80 were sample-exact.
read_frames
Section titled “read_frames”def read_frames(path: str, start: float, count: int, fps: float) -> np.ndarraycount frames of path at start + i / fps seconds, as uint8 [count, H, W, 3] RGB.
Each frame is asked for by its own time: the reader seeks to the first and steps to the rest, where a subclip’s iterator enumerates times from a float duration and can come up one short.