SoundWalk: five seconds of sound, one minute outside
This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass
What I Built
A doorstep, a park bench, a patch of shade. Pick somewhere ordinary and give it a minute. What comes into focus?
SoundWalk turns five seconds of sound into a reason to listen for sixty. Record your surroundings, let a local AI model suggest a listening direction, and choose what to follow. At the end, bring back one sentence in your own words.
I built it around a handoff. The model supplies the starting point. The listener takes over.
For water, the invitation is: “Find one small sound inside the flow.” For street sounds, it asks you to follow something as it approaches and fades, then hear what emerges in the gap afterward. A familiar place becomes a small listening exercise.
Once you start your minute, the page dims. The microphone has already stopped. Read the invitation, look up, and let the place supply the experience. A soft chime brings you back.
Your observation becomes a Sound Card: a little keepsake with the model's suggestion, your chosen direction, and the words you wrote. That is the ending I wanted—a specific detail worth taking home.
Demo
Take a soundwalk → · Watch the 77-second browser walkthrough →
{% youtube 757q9l9g0uQ %}
1. Prepare the local model while you have a connection. The first visit downloads about 18 MB.
2. Find a place outside to pause, then record five seconds.
3. Choose a listening direction and start your minute. Keep the tab open while the screen dims; changing tabs pauses the timer.
4. Write one observation. Save your card in the browser, export a PNG, or share the words.
At your desk, Try a shoreline sample offers a complete rehearsal using a credited sea-wave recording. The walkthrough follows this route with actual local inference, an edited view of the sixty-second timer, and a clearly labeled scripted observation.
!A SoundWalk card showing Water & rain, its licensed-sample source, and a scripted observation.
Sample rehearsal. The card's observation is a scripted example; the credited five-second sea-wave clip loops during the listening minute.
If you try it outdoors, share your phone/browser, the direction you chose, and one thing you noticed. That is the feedback I most want to build on.
Code
Source, local setup, and credits →
SoundWalk was designed and implemented with Codex, OpenAI's coding agent. The application and listening invitations are MIT-licensed. YAMNet and TensorFlow.js are Apache 2.0; their files and attribution ship with the repository.
How I Built It
Turn a classification into a question
YAMNet scores 521 audio-event classes. SoundWalk maps relevant labels into a smaller vocabulary: birds, water, wind, street rhythms, and life nearby. Each group uses its strongest matching label, keeping related labels together.
Then comes the important decision: the person chooses the direction. A bus can dominate five seconds of audio while a quieter rustle holds your interest. Choosing the rustle is already an act of attention. The final card preserves the model's suggestion and your choice as separate fields.
Weak or silence-dominant matches lead to an open invitation: find your nearest sound, find your farthest sound, and move your attention between them.
This is the design work around the AI. “Water” gives the activity context. “Find one small sound inside the flow” gives the listener something to do. The prompts are authored, short, and visible in audio.mjs.
Pack the model before heading outside
Five-second microphone capture
→ mono audio at 16 kHz
→ YAMNet in a browser worker
→ listening suggestions → your choice
→ sixty seconds → your observation
An AudioWorklet captures the audio; the input track stops after five seconds. Resampling prepares it for YAMNet. TensorFlow.js runs inference on the CPU in a Web Worker, keeping the interface responsive.
The model files total about 16.14 MB. Making that first download visible became part of the experience: a preparation screen before the walk.
A service worker caches the app, runtime, model graph, four weight shards, labels, and sample. When the footer reads “Ready for an offline soundwalk,” that browser can reopen the experience offline. Audio processing happens on the device. The latest fifty cards live in browser storage; exported images are ordinary files you can keep.
Recording the demo caught a useful browser wrinkle: the cached WAV decoded for inference, while native media playback rejected its URL. Creating a local Blob from the same bytes fixed playback, including after an offline reload. A playback failure now pauses the rehearsal and offers a Resume action. Hearing the sample is part of making the invitation work.
Give development a repeatable shoreline
The bundled sample made the whole sequence reproducible. In Chromium 153 on Linux, its first YAMNet inference took 4.23 seconds on the CPU and led to the water invitation. The rehearsal completed sixty seconds, saved a card across a reload, and exported a PNG.
An offline run exercised the microphone path with Chromium's file-backed audio simulation. Selecting Wind after the model suggested Water checked the handoff I cared about: the saved card retained both. The setup and results are documented here.
The sample comes from HerbertBoland's “Branding_kort.wav”, via ESC-10, under CC BY 3.0. Its source label follows it through the rehearsal and saved card.
Why Does Open Innovation Matter?
Open weights made the product's geography possible: the model travels with you. Once prepared, SoundWalk can do its work on the listener's device, wherever that browser is taken.
Open code makes the next layer just as accessible. You can trace an audio label into a category, read the invitation it suggests, and change the experience in a small edit. A school-garden version could keep the same model and rewrite the invitations around its own sounds and language.
That is the kind of contribution I would love to see: someone who knows a place writing a better question to ask there.
The open model supplies a shared starting point. Every walk ends with something particular: one person's attention, in one place, for one minute.
Prize Categories
Best Use of GitHub Copilot — GitHub Actions automation. The category explicitly includes “automate your project with GitHub Actions.” SoundWalk's workflow runs audio tests, checks licensing provenance and required model assets, and verifies the offline dependency pack before deploying to Pages.
That gate protects the experience described above: a missing weight shard stops publication. The release verification and deployment is public.
1 comment
Outside: a private streak app that listens to your walk with open-source AI
This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass
What I Built
Outside is a small, private Android app that gets me off the screen and into the world. I tap "I'm outside", put the phone in my pocket, and walk. While I'm out, the app listens to the sounds around me and writes them into a journal, so a walk becomes "Birds, water, wind" instead of just "20 minutes."
- Streak with rest days: an animated flame counts consecutive days outside. You earn one rest day for every 7 days outside, so one missed day doesn't wipe out your streak.
- Automatic sound journal: it labels sounds in 5-second windows (birds, wind and leaves, water, traffic, voices, animals, bells, footsteps) and saves only the labels.
- Approximate step count from the phone's motion sensor.
- Time-of-day sky: the header changes from dawn to night with the clock.
- Journal as text: any walk can be exported as plain text that I own.
It is for anyone who wants a gentle reason to go outside. It is deliberately not social: no account, no feed, no photo posting, no server. The screen is only used for a few seconds at the start and end of a walk.
Demo
{% embed youtube.com %}
!Image description
!Image description
Code
Nirajkam/outsideapp_week1challage_hacktoberfest
How I Built It
- Open-weight model: YAMNet by Google (Apache 2.0), which recognizes 521 sound classes. It runs fully on the phone through TensorFlow.js, with the model files bundled inside the app.
- App: plain HTML, CSS, and JavaScript, wrapped as an Android app with Capacitor. Everything is stored in local storage on the device.
- Turning raw classes into a journal: I grouped the 521 classes into 8 friendly categories. Steady sounds (water, wind, traffic) are scored by their average over a window, and short sounds like birdsong by their peak.
- Listening loop: it starts when a walk starts and stops when it ends. It keeps the screen awake so the microphone keeps working, and classifies 5-second windows in memory.
- Debug view: the app shows the model's top 3 raw guesses on screen, which made tuning much easier.
What I Learned From Testing
I tested by playing sounds near the microphone. Speech was picked up and labeled as "Voices". My first waterfall test was not detected at all. The model's raw classes for running water were missing from my "water" group, and my detection threshold was too high for a steady sound. I added more water-related classes (rain on surface, waves, gush, spray), lowered the water threshold, and switched steady sounds to window averages.
Limits I want to be upfront about:
- It labels sound types, not species. It says "Birds", not "robin".
- It only listens while the app is open and the screen is on, which costs battery.
- Steps are an estimate from the accelerometer, not a real pedometer.
- Android only for now.
Why Does Open Innovation Matter?
- Privacy: microphone audio never leaves the phone. It's classified in memory for a few seconds and then discarded, and only the labels are saved. With a closed cloud audio API, recordings of my walks would go to someone else's server.
- It works with no signal: the model ships inside the app, so it can work on a trail with no coverage, which is where an outdoor app is most useful.
- Free to run: no API keys, no per-request cost, no subscription.
- I could change it: the model is a plain TensorFlow.js file and the category mapping is a JSON file. When water detection failed, I fixed it by editing the mapping and thresholds, which I couldn't have done with a closed black-box API.
Prize Categories
This project uses YAMNet, not Gemma, so I'm not entering Best Use of Gemma. I'm entering the overall prize.
Mentions
SoundWalk: five seconds of sound, one minute outside This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass What I Built A doorstep, a park bench, a patch of shade. Pick somewhere ordinary and give it a minute. What comes into focus? SoundWalk turns five seconds of sound into a reason to listen for sixty. Record your surroundings, let a local AI model suggest a listening direction, and choose what to follow. At the end, bring back one sentence in your own words. I built it around a handoff. The model supplies the starting point. The listener takes over. For water, the invitation is: “Find one small sound inside the flow.” For street sounds, it asks you to follow something as it approaches and fades, then hear what emerges in the gap afterward. A familiar place becomes a small listening exercise. Once you start your minute, the page dims. The microphone has already stopped. Read the invitation, look up, and let the place supply the experience. A soft chime brings you back. Your observation becomes a Sound Card: a little keepsake with the model's suggestion, your chosen direction, and the words you wrote. That is the ending I wanted—a specific detail worth taking home. Demo Take a soundwalk → · Watch the 77-second browser walkthrough → {% youtube 757q9l9g0uQ %} 1. Prepare the local model while you have a connection. The first visit downloads about 18 MB. 2. Find a place outside to pause, then record five seconds. 3. Choose a listening direction and start your minute. Keep the tab open while the screen dims; changing tabs pauses the timer. 4. Write one observation. Save your card in the browser, export a PNG, or share the words. At your desk, Try a shoreline sample offers a complete rehearsal using a credited sea-wave recording. The walkthrough follows this route with actual local inference, an edited view of the sixty-second timer, and a clearly labeled scripted observation. !A SoundWalk card showing Water & rain, its licensed-sample source, and a scripted observation. Sample rehearsal. The card's observation is a scripted example; the credited five-second sea-wave clip loops during the listening minute. If you try it outdoors, share your phone/browser, the direction you chose, and one thing you noticed. That is the feedback I most want to build on. Code Source, local setup, and credits → SoundWalk was designed and implemented with Codex, OpenAI's coding agent. The application and listening invitations are MIT-licensed. YAMNet and TensorFlow.js are Apache 2.0; their files and attribution ship with the repository. How I Built It Turn a classification into a question YAMNet scores 521 audio-event classes. SoundWalk maps relevant labels into a smaller vocabulary: birds, water, wind, street rhythms, and life nearby. Each group uses its strongest matching label, keeping related labels together. Then comes the important decision: the person chooses the direction. A bus can dominate five seconds of audio while a quieter rustle holds your interest. Choosing the rustle is already an act of attention. The final card preserves the model's suggestion and your choice as separate fields. Weak or silence-dominant matches lead to an open invitation: find your nearest sound, find your farthest sound, and move your attention between them. This is the design work around the AI. “Water” gives the activity context. “Find one small sound inside the flow” gives the listener something to do. The prompts are authored, short, and visible in audio.mjs. Pack the model before heading outside Five-second microphone capture → mono audio at 16 kHz → YAMNet in a browser worker → listening suggestions → your choice → sixty seconds → your observation An AudioWorklet captures the audio; the input track stops after five seconds. Resampling prepares it for YAMNet. TensorFlow.js runs inference on the CPU in a Web Worker, keeping the interface responsive. The model files total about 16.14 MB. Making that first download visible became part of the experience: a preparation screen before the walk. A service worker caches the app, runtime, model graph, four weight shards, labels, and sample. When the footer reads “Ready for an offline soundwalk,” that browser can reopen the experience offline. Audio processing happens on the device. The latest fifty cards live in browser storage; exported images are ordinary files you can keep. Recording the demo caught a useful browser wrinkle: the cached WAV decoded for inference, while native media playback rejected its URL. Creating a local Blob from the same bytes fixed playback, including after an offline reload. A playback failure now pauses the rehearsal and offers a Resume action. Hearing the sample is part of making the invitation work. Give development a repeatable shoreline The bundled sample made the whole sequence reproducible. In Chromium 153 on Linux, its first YAMNet inference took 4.23 seconds on the CPU and led to the water invitation. The rehearsal completed sixty seconds, saved a card across a reload, and exported a PNG. An offline run exercised the microphone path with Chromium's file-backed audio simulation. Selecting Wind after the model suggested Water checked the handoff I cared about: the saved card retained both. The setup and results are documented here. The sample comes from HerbertBoland's “Branding_kort.wav”, via ESC-10, under CC BY 3.0. Its source label follows it through the rehearsal and saved card. Why Does Open Innovation Matter? Open weights made the product's geography possible: the model travels with you. Once prepared, SoundWalk can do its work on the listener's device, wherever that browser is taken. Open code makes the next layer just as accessible. You can trace an audio label into a category, read the invitation it suggests, and change the experience in a small edit. A school-garden version could keep the same model and rewrite the invitations around its own sounds and language. That is the kind of contribution I would love to see: someone who knows a place writing a better question to ask there. The open model supplies a shared starting point. Every walk ends with something particular: one person's attention, in one place, for one minute. Prize Categories Best Use of GitHub Copilot — GitHub Actions automation. The category explicitly includes “automate your project with GitHub Actions.” SoundWalk's workflow runs audio tests, checks licensing provenance and required model assets, and verifies the offline dependency pack before deploying to Pages. That gate protects the experience described above: a missing weight shard stops publication. The release verification and deployment is public.
Outside: a private streak app that listens to your walk with open-source AI This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass What I Built Outside is a small, private Android app that gets me off the screen and into the world. I tap "I'm outside", put the phone in my pocket, and walk. While I'm out, the app listens to the sounds around me and writes them into a journal, so a walk becomes "Birds, water, wind" instead of just "20 minutes." - Streak with rest days: an animated flame counts consecutive days outside. You earn one rest day for every 7 days outside, so one missed day doesn't wipe out your streak. - Automatic sound journal: it labels sounds in 5-second windows (birds, wind and leaves, water, traffic, voices, animals, bells, footsteps) and saves only the labels. - Approximate step count from the phone's motion sensor. - Time-of-day sky: the header changes from dawn to night with the clock. - Journal as text: any walk can be exported as plain text that I own. It is for anyone who wants a gentle reason to go outside. It is deliberately not social: no account, no feed, no photo posting, no server. The screen is only used for a few seconds at the start and end of a walk. Demo {% embed youtube.com %} !Image description !Image description Code Nirajkam/outsideapp_week1challage_hacktoberfest How I Built It - Open-weight model: YAMNet by Google (Apache 2.0), which recognizes 521 sound classes. It runs fully on the phone through TensorFlow.js, with the model files bundled inside the app. - App: plain HTML, CSS, and JavaScript, wrapped as an Android app with Capacitor. Everything is stored in local storage on the device. - Turning raw classes into a journal: I grouped the 521 classes into 8 friendly categories. Steady sounds (water, wind, traffic) are scored by their average over a window, and short sounds like birdsong by their peak. - Listening loop: it starts when a walk starts and stops when it ends. It keeps the screen awake so the microphone keeps working, and classifies 5-second windows in memory. - Debug view: the app shows the model's top 3 raw guesses on screen, which made tuning much easier. What I Learned From Testing I tested by playing sounds near the microphone. Speech was picked up and labeled as "Voices". My first waterfall test was not detected at all. The model's raw classes for running water were missing from my "water" group, and my detection threshold was too high for a steady sound. I added more water-related classes (rain on surface, waves, gush, spray), lowered the water threshold, and switched steady sounds to window averages. Limits I want to be upfront about: - It labels sound types, not species. It says "Birds", not "robin". - It only listens while the app is open and the screen is on, which costs battery. - Steps are an estimate from the accelerometer, not a real pedometer. - Android only for now. Why Does Open Innovation Matter? - Privacy: microphone audio never leaves the phone. It's classified in memory for a few seconds and then discarded, and only the labels are saved. With a closed cloud audio API, recordings of my walks would go to someone else's server. - It works with no signal: the model ships inside the app, so it can work on a trail with no coverage, which is where an outdoor app is most useful. - Free to run: no API keys, no per-request cost, no subscription. - I could change it: the model is a plain TensorFlow.js file and the category mapping is a JSON file. When water detection failed, I fixed it by editing the mapping and thresholds, which I couldn't have done with a closed black-box API. Prize Categories This project uses YAMNet, not Gemma, so I'm not entering Best Use of Gemma. I'm entering the overall prize.