{"id":229,"date":"2017-09-09T00:38:09","date_gmt":"2017-09-09T06:38:09","guid":{"rendered":"http:\/\/www.blog.jay-greco.com\/wp\/?p=229"},"modified":"2017-09-09T00:38:09","modified_gmt":"2017-09-09T06:38:09","slug":"downloading-saving-a-nest-cam-live-stream-using-a-raspberry-pi-debian-linux","status":"publish","type":"post","link":"http:\/\/www.blog.jay-greco.com\/wp\/?p=229","title":{"rendered":"Downloading & Saving a Nest Cam Live Stream Using a Raspberry Pi + Debian Linux"},"content":{"rendered":"

Tonight, I stumbled on an interesting post on Reddit<\/a>. It linked to two Nest Cams livestreaming the landfall of hurricane Irma from a Miami condo. I popped the streams open on my phone (the hurricane had not hit yet, and the sky was mostly clear) and thought about the fact that I was probably going to fall asleep before the power went out and killed the stream.<\/p>\n

Being an incredibly good-looking and overconfident dweeb, I then thought about the fact that there’s got to be a good way to rip and save a Nest Cam livestream so that I could watch it tomorrow, maybe post a time-lapse, and gain all of the Karma. After all, its already been done with tons of other streams from various sites. It seemed like a decent Friday night hackathon in the making, and would at least solve the problem of me falling asleep lest I fail.<\/p>\n

I decided I wanted something that was (1) automatic, and (2) running in the background so that I didn’t need to stay up all night or keep my computer on. I ran through the list of options in my head:<\/p>\n

    \n
  1. Just stream them in a browser window and use screen capture (lame and fails both #1 and #2),<\/li>\n
  2. Use one of the millions of Chrome or Firefox plugins that allows for saving of streams (extra lame and still fails #2),<\/li>\n
  3. Use some sort of stream-ripping software built for Linux so I could load it on my always-running Pi (not lame, but impossible for me to find something that worked after looking for an hour or so), or<\/li>\n
  4. Hack it and do it myself.<\/li>\n<\/ol>\n

    If you haven’t guessed already, I went with #4. I’m going to show you how I figured it out and how to do it yourself. This assumes some basic knowledge of Linux command line and shell scripting.<\/p>\n

    First thing first, I loaded one of the Nest Cam streams using the links provided on Reddit. The video livestream itself sits inside a Nest-branded HTML page that does this really annoying thing where it auto-pauses and pops over a Nest advertisement every once in a while. If I wasn’t going to rip it out already, I would have been sufficiently annoyed by this to figure out how to get to the base stream.<\/p>\n

    \"\"<\/a><\/p>\n

    I poked around inside the page source using the Safari dev tools to see if I could find any obvious stream container or link, but didn’t see anything. I did find a more minimal stream that is formatted for Twitter but it still does the popover thing. Boo.\u00a0I also poked around in the javascript (warning: there’s a lot) to see if the stream was being lazy-fetched from any obvious source. Again, nothing. Boo.<\/p>\n

    I decided to use the Timelines tool to see what’s being loaded on the network. I recorded for a few seconds and saw what was clearly a periodic fetch taking place. There’s an XHR request going out approximately every 4 seconds. It’s loading a media_xxxxxxxx_123.ts<\/code> file and a chunklist_xxxxxxxx.m3u8<\/code> file after each request. This is definitely an MPEG-2 stream, with the chunklist serving as a manifest for the media.ts file.\u00a0Bingo!<\/p>\n

    \"\"<\/a><\/p>\n

    .m3u8 files are commonly used to define video streams, and so I knew I was on the right track. Right-clicking on the m3u8 file and choosing “Copy Link Address” and pasting it into the Safari address bar yielded a base-level video stream with no extra junk (*cough*) on top of it. It looks like Nest streams their livestream content from stream-bravo.dropcam.com<\/code>\u00a0or from stream-delta.dropcam.com<\/code>. (Both are currently using\u00a0Wowza Streaming Engine 4 Subscription Edition 4.7.1 build20635)<\/p>\n

    \"\"<\/a><\/p>\n

    The next step was saving the stream using this URL. Time to break out the Pi! I figured I could use ffmpeg to do this, and after a quick google search, my assumptions were confirmed. This<\/a> StackOverflow question gave me what I needed, except I wanted to ensure that the ffmpeg command was always running (in the event the stream broke up and was restarted, a network issue occurred, etc).<\/p>\n

    For those of you who just want to save a Nest Cam stream to disk using Raspbian\/Raspberry Pi\/Debian\/Other Linux, this is the command that will do it for you (you need ffmpeg<\/code>\u00a0installed in order to use this): ffmpeg -i http:\/\/your_stream_chunklist_link.m3u8 -c copy -bsf:a aac_adtstoasc \/path\/to\/output\/file.mp4<\/code>. For example, this is the command I used to save the stream I was watching to my home directory: ffmpeg -i https:\/\/stream-delta.dropcam.com\/nexus_aac\/a8a645a10ef24a50b250c14a08b02ef9\/chunklist_w719996219.m3u8 -c copy -bsf:a aac_adtstoasc Stream.mp4<\/code><\/p>\n

    In order to make sure that ffmpeg was always restarted in case of any issues, I whipped up the following shell script (named runStream.sh<\/code>) to be run as a cronjob:<\/p>\n

    #!\/bin\/bash\r\n#make-run.sh\r\n#make sure a process is always running.\r\n\r\nprocess=LivingRoom\r\nnow=$(date +%Y%m%d%H%M%S)\r\nmakerun=\"ffmpeg -i https:\/\/stream-delta.dropcam.com\/nexus_aac\/a8a645a10ef24a50b250c14a08b02ef9\/chunklist_w719996219.m3u8 -c copy -bsf:a aac_adtstoasc \/media\/HDD\/Stream_$now.mp4\"\r\n\r\nif ps ax | grep -v grep | grep $process > \/dev\/null\r\nthen\r\n exit\r\nelse\r\n $makerun &\r\nfi\r\n\r\nexit<\/pre>\n
    The script checks to see if the ffmpeg command is running using ps ax<\/code>\u00a0and grep<\/code>. If it is, there is no need to start it, so it exits. If it isn’t, the script is started using the makerun<\/code>\u00a0shell command. Note the $now<\/code>\u00a0variable at the end of the filename: it automatically appends a puncuation-less timestamp to each video file, so that the previous file is not lost when ffmpeg is automatically restarted.<\/p>\n
    The last thing to do was to make the script executable using chmod +x runStream.sh<\/code>\u00a0and add it to the crontab using crontab -e<\/code>. I set it to run every minute (can’t miss any of the action!) using the following crontab:<\/p>\n
    # m h\u00a0 dom mon dow \u00a0 command\r\n* * * * * \/home\/pi\/runStream.sh<\/pre>\n
    After saving the changes and waiting a minute, I saw the first video file pop up. After running for a few hours, the auto-restart was a great idea, because it’s kicked in several times (likely due to haphazard internet because there a HURRICANE).<\/p>\n
    Stay safe out there, Florida. It’s going to get crazy.<\/p>\n","protected":false},"excerpt":{"rendered":"
    Tonight, I stumbled on an interesting post on Reddit. It linked to two Nest Cams livestreaming the landfall of hurricane Irma from a Miami condo. I popped the streams open on my phone (the hurricane had not hit yet, and the sky was mostly clear) and thought about the fact that I was probably going … <\/p>\n
    Continue reading “Downloading & Saving a Nest Cam Live Stream Using a Raspberry Pi + Debian Linux”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":232,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"quote","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false},"version":2}},"categories":[9,18],"tags":[87,89,92,88,90,86,91,25],"class_list":["post-229","post","type-post","status-publish","format-quote","has-post-thumbnail","hentry","category-projects","category-reverse-engineering","tag-cam","tag-dropcam","tag-ffmpeg","tag-google","tag-live-stream","tag-nest","tag-nest-cam","tag-reverse-engineering","post_format-post-format-quote"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/www.blog.jay-greco.com\/wp\/wp-content\/uploads\/2017\/09\/Screen-Shot-2017-09-08-at-11.46.00-PM.png?fit=1440%2C447","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p755Zm-3H","jetpack-related-posts":[],"_links":{"self":[{"href":"http:\/\/www.blog.jay-greco.com\/wp\/index.php?rest_route=\/wp\/v2\/posts\/229"}],"collection":[{"href":"http:\/\/www.blog.jay-greco.com\/wp\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.blog.jay-greco.com\/wp\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.blog.jay-greco.com\/wp\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.blog.jay-greco.com\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=229"}],"version-history":[{"count":2,"href":"http:\/\/www.blog.jay-greco.com\/wp\/index.php?rest_route=\/wp\/v2\/posts\/229\/revisions"}],"predecessor-version":[{"id":234,"href":"http:\/\/www.blog.jay-greco.com\/wp\/index.php?rest_route=\/wp\/v2\/posts\/229\/revisions\/234"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/www.blog.jay-greco.com\/wp\/index.php?rest_route=\/wp\/v2\/media\/232"}],"wp:attachment":[{"href":"http:\/\/www.blog.jay-greco.com\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=229"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.blog.jay-greco.com\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=229"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.blog.jay-greco.com\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=229"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}