πŸ‘ˆ

HOT The word new as would be seen on a sticker on an item in store inventory trying to attract new customers. Tone

Tone?

Written on October 28, 2020

Tone.js is self described as, "a Web Audio framework for creating interactive music in the browser." It is an abstraction built on top of the Web Audio API to make it easier to create music with javascript!

We became interested in Tone.js because:

We were invited to collaborate with JS Party, a javascript podcast, but the requirement was we needed upgrade the audio of our show. With audio on the mind we wanted to explore a javascript library that relates to audio which also was a gateway into a larger conversation about audio on the web in 2020.

Check out the episode here πŸ‘‡

Screenshot from tone.js website landing page

What We Built

Reading the tone.js documentation inspired us to create an audio hover state. We initially explored a recording of a laugh playing on hover, but we ended up creating audio previews of our episodes for our explored section on the home page.

Notes from the future

In the show we talk vaguely and nostalgically about when "audio was unrestricted on the internet." While working with tone.js, we were a bit surprised that a user needs to opt in to an audio experience. Apparently you can't build music to auto play on your site.

After recording, we researched further to see at what point did the web audio spec change to stipulate that a user needs to intentionally opt into a web audio experience.

It turns out the spec never changed, it was the browser rules that changed.

For example, the actual commit for Firefox 65 that was released on January 29th 2019 that "postpones running the Audio Context."

A snippet from the mozilla commit that changed the auto play rules for audio.

And a little bit more from google chrome about their auto play changes.

A snippet from the chrome developer blog that turned off audio play

Thanks to Chris Wilson from Google and Paul Adenot from Mozilla for that information.

Screenshot of our javascript that launches the audio hover states

Transcript

What follows is the transcript for our tone.js epiosde!

Rushali: Yeah, we're not going political on this podcast...

Judith: I think we already went political.

Rushali: Oh shii…

(Intro)

Judith: Hey, it's Rushali and Judith, and we're here with Hot New Tech. We're going to start with a story. So Rushali insisted that we joined Twitter a couple of weeks ago, and I'm very resistant because I'm scared of Twitter because I follow some mean people on Twitter. But then after 10 minutes of being on Twitter, we were invited to collaborate with JS Party, which is my favorite JavaScript podcast, super exciting, but there was a requirement that we upgrade our audio. We just invested in new microphones.

Rushali (00:55): We have the very cute black colored blue Yetis. I don't know if those are good microphones, but now we have them. And speaking of audio, we were wondering, how do we combine audio and JavaScript? Because this episode is in collaboration with JS Party. So the perfect library that we thought we would test out is tone.js, which is a library based on top of web audio API, and the way that the library describes itself, it calls itself a web audio framework for creating interactive music in the browser.

Judith (01:38):For our piece we are going to do a simple audio on hover. Another thing since we were going to be on JS Party, I was listening to their show and I noticed how they use a lot of audio aspects in their show. I was thinking, it'd be interesting to automate aspects of the audio. For example, like you have data points about your show, like how many listeners you have, or, if you're getting comments maybe that could be automated into your jingle. I don't know. I think there's this tension between art and engineering and with engineering, there is focus on automation and efficiency and with art, there's a lot of focus on expression. And I feel like this kind of bridges the gap between the two where we bringing more automation into arts and we can bring more expression into engineering. And this kind of feels like there's a lot of opportunity there to do that.

Rushali (02:22): Definitely both those fields start to lose their sense when they focus on money. So when engineering focuses too much on automation, you end up in a world where, you're treading on the line of being slightly unethical, maybe. And when art expressiveness goes into making money, again, you again are in this weird place of - what is this art doing?

Judith (02:54): Yeah, we, I think we always diverge at this point because I am essentially a capitalist in the sense that I think it's the best option so far. I think that, with monetizing, you're forcing yourself to bring value to other people. Whereas if you're making art from a "more pure place", it could really not speak to anyone, you know, because maybe it speaks to you, but if it's not speaking to other people, then is there a point in that? I mean, there is a lot of cases of artists who, you know, were so ahead of their time that if they paid more attention to what other people wanted, maybe they would not have made as much groundbreaking work. I can see that argument. But for the most part, I do think that a focus on bringing value will force the best out of you. However, I agree with you that you can also force the worst out of you. So it's definitely there. It wouldn't even be a conversation if there wasn't an argument to make on both sides.

Rushali: It's a balance. At the end of the day, I feel like I'm not a capitalist. I'm not a socialist. I would try it on the lens of social capitalism, if that's a thing. And that just means balancing things out, don't make so much money that you're leeching it off of people, but don't be so idealistic that you don't make any money. Yes, bring value to people. Definitely do things in moderation, be balanced.

Judith (04:15): I don't think our country remembers what that is anymore. Yeah.

Rushali (04:17): Oh, we're not going political on this.

Judith (04:19): I think we already went political, but anyways, back to music!

Rushali (04:24):Back to music back to JavaScript. So how it works? Tone.js is built on top of web audio API and web audio API would allow you to play sounds and music and do other things, but tone.js just makes it easier because they have these methods for oscillators and synthesizers. I do want to say that my experience with JavaScript began with tone and another library called p5.js, which is also more of a visual artsy library and being introduced to JavaScript through tone makes me a weird engineer. The output of all of my coding was audio. So it was an interesting journey.

Judith (05:09): I have never seen the library before. So my experience was just reading the docs. And the first code example on the docs teaches you how to play a middle C for the duration of an eighth note. And the code looks like synth.triggerAttackRelease, which obviously takes some learning for me to know what it is doing. And then you enter in the note, which is C4, and then you enter the duration of the note, which is an eighth note. It is really interesting that it goes as far as timing with notes, note language and not seconds language. I think it's cool. It's asserting the music paradigm into the code. I found that really interesting. Did Rushali mention that you can use time intervals as well?

Rushali: (06:00):Yeah. You can change the way that you're using time. So you can use one eighth of a note. You can use seconds as well. Yeah. There's a lot of musical knowledge that you might want while using tone. And that is great. Documentation explains things like what triggerAattackRelease is and what synthesizes and oscillators are, but that does mean that you need to learn music and coding at the same time.

Judith (06:28): So we started by getting the C4 note to work, and as we started to work on it, we realized that the way that we could use it on our site is that for certain elements, we can have a pre-recorded audio play. But when we tried that, the hover didn't work because the user has to engage with something before the audio plays. So we have a little button now that says, enable weird audio sounds, and you can click on it. And then once you click on it, then the hover states work. And now when you hover over the exports section on desktop, you'll hear something fun. So speaking of funny time sucks, we have this free audio library that we use a lot for jingles and stuff that we can link in the notes. And I love it. And we looked at almost all of their laughter sounds and they all were so... what's the opposite of genuine? Canned, fake, fake. It was actually really hard to find a good laughter sound. We might have to use our own because we do laugh a lot.

Rushali (07:35): Maybe we should ask our listeners to send in their laughters to us.

Judith (07:40): Uuuuuuuh! That's a good one!

Rushali (07:41): So we can have this little repository of genuine laughters and not fake laughters.

Judith (07:47): A repository of genuine laughter. It sounds like my favorite thing in the world.

Rushali (07:54): He-heh-he-heh...

Judith (07:56):So in essence, how it works, you import the tone.js library, and it's a layer on top of the web audio API library, but it makes it easier and you need to create a player. And there's two directions from there. You can use external audio that's prerecorded, or you can create your own music with these trigger attack releases and notes and times, but we sort of stuck to the pre-recorded audio because that's much simpler. We're not musicians. And that was pretty simple. And you can do that probably in 10 minutes, the sort of creating audio with the library is obviously more complicated. You would require some musical knowledge, but there is significant documentation and examples. So it's definitely possible and very cool.

TLDR Ratings

Ease

Rushali (08:56): Was it easy to use?

Judith (08:58): I would give it a 7. I think it's a complicated one. And I think that reflects the two directions of the library. I don't think it's easy to create music, period. Um, so this is not a magical wand.

Rushali (09:11): I think we should say that it's not easy to create good music. You can make bad music very easily.

Judith (09:17): Interesting. I'm thinking, I'm trying to decide if that's true or not in my head, even creating bad music sounds hard because it's like a level of vulnerability there. So I would say even bad music is maybe even harder because you're more embarrassed. Fine. Okay. I'm going back to the 7 because it was doable and there was significant enough examples and documentation. So certainly doable, but it's not like it's not like writing one, two, three on a paper. It's not a 10. It is a 7!

Rushali (09:47): For ease? I would give it a 7 or an 8. 7. I would give it a 7 too. You know why? Because I have a biased opinion with this because I started this in 2017. So I have seen the docs before, I've done projects with it and I learned JavaScript with it. So I can't really give it like a non-biased rating and my biased rating is 7 out of 10.

Docs

Rushali (10:15): For documentation I would give it an 8 out of 10. Again, this is a biased opinion.

Judith (10:20): I was leaning more towards the 7 out of 10, but I do think because I like things to be spoonfed to me and I want to be able to copy and paste everything. So that's my definition of a 10. Copy and paste did not work. And there were some little glitches with the Javascript and with attaching to Dom objects. However I would say that the juice is in the GitHub because there's a million examples there that work. I think it's underrated when a demo actually works. How many times have you looked up a library and you're like totally sold and then click on the demo page and the demo's broken? And you're like, of course I'm not going to use this library, the demo is broken. So I have to say that the fact that all the demos in the tone.js library work, that's almost a 10 right there. And then you can look into the code and that could even be copy and paste for the most part. So, you know, I'm going to meet you on the 8 because there are so many working demos that are awesome. So I'll meet you on the 8.

Fun

Judith (11:21): I'm going to give it a 9 out of 10 because when I hovered on that button and it laughed, I laughed. It's so simple, but it just made me laugh. That's a 9 for me.

Rushali (11:34): That was really a great moment to watch you hover and laugh too. That laugh was just so satisfying, also for just a viewer.

Judith (11:43): I think the delight of the Internet has really gone down the drain and it kind of reminds me of the nineties or like when the internet was just strange and delightful. Yeah. And maybe that's just part of when audio was allowed and not restricted... They didn't know it was a better place. I don't know. When's the last time you laughed on a hover?

Rushali (12:00):Yeah, exactly. When was the last time you laughed at hover state is the question? We should all be asking ourselves in 2020.

Heat

Judith: I think that one might initially give it a low rating because of the low market value of audio on the web. However, my argument would be that because data, I would say data is the hottest thing on the market right now. Everything's about data, data visualization, data maximization, data privacy, data, data, data, data, data, it's all about data. But I think that data sonification has lot of potential. Um, and there you have it. You can do that with this library.

Rushali (12:44): I agree with all of the parts that you said, what is concerning to me is that when I graduated, I learned javascript through tone.js and p5.js and when you apply for a job, if you know React, they're like, yes, you're hired. And then when they go like what stuff do you know? And I went like I know tone. And they were like, what the f*ck is tone? So it was very hard when I was job hunting, but I had so much fun while learning it. So, you know, you win some, you lose some. And, uh, I wouldn't learn, I wouldn't want to learn JavaScript with React -that just sounds very boring. 7 out of 10. Like, I can't give it more than 7 because , if there's a student who is learning how to code right now, listening to this and goes, Oh, I'm going to try to own it and add it to my resume. I don't know how many companies are going to be like, Wow, what a great library to learn. That's not what companies are looking for, unless it's a specific audio company, which is also amazing. But if you're into open source stuff, then yeah, right, yeah. Use this library. It's fun. It's hard to a certain degree. And it's fun.

Judith (14:05): I definitely would agree with you that it shouldn't be the only JavaScript library on your resume, but I think it's a hot tool to have in your box.

Rushali (14:13): Oh, for sure. And if you're a musician, it's like way too hot. It's like what it would be like, if a musician walked up to me and was like, I can code music, I'd be like, ha-ha-ha, this is good.

Judith (14:28): Clap, clap, clap. Yeah. So can we bump you up? We have to obviously have an average. So you say 7. I prefer even numbers. Can I sell you on the 8?

Rushali (14:44): Fine, 8 it is.

How Long It Took

Judith (14:47): To get that C-sharp going, you're looking at 15 minutes. To to get our hover states working before we sort of knew about the audio opt in, I would say like an hour just to, you know, also we had a CORS error with local audio. We had to spin up a server. So there's that little, there's a couple of small challenges, which were always going to happen with the new thing. So anyways, I would say an hour to get something custom working. And then to actually create something with a custom song? How much, since you I've never done that, how long do you think it would take to make a real song?

Rushali:(15:26): Are we musicians or not?

Judith (15:30): How long did it take you to get your first real project going with tone?

Rushali: A day

Judith: Wow.

Rushali (15:37): But I, I like, I didn't know anything, like, I don't know what triggerAttack really is.

Judith (15:41): Right. But that's how it is. That's how people are. When they come into this they don't know anything. Are you saying if you're an audio engineer coming into it?

Rushali (15:47): Yeah. If you know a lot of these concepts, you would know like, Oh, I think if I played, F or what is it? Music theory. Like if she knew what notes to play afterwards. I just make noise at the end of it

Judith (15:59): An hour. That is how long it took for us to get something that we wanted working.

Rushali (16:04): It's time for our pros and cons.

Judith (16:06): So, pros. The first pro is it makes the Internet delightful again. As we asked - when's the last time you laughed on a hover state? If you've seen the movie "Social Dilemma", one thing the movie mentions is that the internet has become like Walmart or a shopping mall. It just feels like everyone's using the same five platforms, WordPress, Shopify, Bootstrap, Material, everything is geared towards efficiency and monetization and engaging the user as long as possible. And following you around and making sure you check out and come back to your checkout and it's become a creepy place. And I think this is a very different direction of making it delightful and fun and experimental. So this is definitely the direction I want the Internet to move in. It seems very silly and frivolous, but if the Internet isn't delightful, people are going to stop using it.

Judith (16:58): If it becomes creepier than it is useful. There's potential accessibility wins. Again. We have done some work with the voice and I think voice has a lot of power. Um, it's a whole new dimension of accessing information. And for example, let's say you had a hover state. There is a screen reader native to the browser, but it's so awkward. What if instead of using a screen reader, you you could enable an audio section where you hover on something and it narrates to you in a human voice with inflection, with, you know, expression. Like I was thinking of redoing my personal website and having just a section with pictures that you hover over. And I tell you a story. That would be very accessible to certain people who don't like reading. So, okay. Accessibility wins. Another thing again, a different dimension of viewing the Internet. We talked about that. It's a good gateway JavaScript library into coding. And lastly, it's an inspiring storytelling tool that I'm going to think of every time now when I have a client. And for example, I get a lot of people wanting data visualization, right. But why not Data sonification, too? What if I add a little bit of audio there as well? It could not be that much effort and be a really rewarding storytelling experience. Those are my pros.

Rushali (18:13): Nice. For the cons. We think that people possibly might be hating audio on the web. I don't know. That is what you all think. Because back in the nineties, the Internet was filled with weird sounds. And now we don't have as much, like the only sound that I think of is the Netflix sound when I'm a show starts. Dum Dum! It's like, that's the one, that's the one that's in my head in 2020. So it's time to bring audio back and make it cool again. But apparently we all hate audio on the internet. So yeah, that would be a con if people hear audio. I like it. But people might not. The second con is that you need to click a button to start. You need an interaction that says, Hey, do a web audio. And, uh, probably cause it was causing accessibility issues. Like you don't want to go on a website and suddenly it starts booming sounds you. So that could be a pro and not just a con I don't know. It could go either way. I'm not doing very well with the cons section

Judith (19:32):No. Cons are great. It's not so black and white.

Rushali (19:34): Yeah. Okay. Yeah, exactly. They're good cons. They are not bad cons. And the lost con might make a little more sense - it doesn’t have a very high market value. The fourth con is it did not turn Judith into a musician. She wanted to become David Guetta and she is not David Guetta yet.

Judith (19:58): And also the automatic feedback, which was cool. I could see it being a cool tool to learn music.

Rushali (20:04): Uuuuuh. That is cool. Learning music.

Judith (20:05): Right. But it didn't automatically inject music knowledge into my head, which is always my goal. Just to Matrix style, load the knowledge. That's what I want.

Would you recommend this

Rushali: One word answer. Yes or no?

Judith (20:18): Yes. I would recommend it.

Rushali (20:25): Thanks. Me too.

Judith: Are we defending our opinions anymore or just stating them?

Rushali: We should defend our opinions.

Judith (20:32): I would recommend it. Yeah. I think there's no reason not to do it. It was pretty fast to learn. And it's a great tool. It's a very diverse tool. I think I would think of your tool set as a portfolio. And this would definitely diversify the portfolio. Right. Cause it's not a single page application library. Right? So if you already know a single page application library or framework, then this is a cool bells and whistles library. I love bells and whistles because clients love bells and whistles. So yeah. Um, I like making clients happy, so I like bells and whistles. And it's cool. It's fun. My only thing is what you mentioned was the fact that people hate audio on the internet, but the fact that you opt into it, you know, kind of mitigates that.

Rushali (21:17): I recommended it because it was fun. That's it? I have no other ones on it, but that's it. If it's fun, use it. If it's not fun, what's the point of it?

Statistics

Judith (21:30): So last time we did our first statistics section and we're going to now. We'd love to hear if you guys are interested in this because we obviously find it very interesting. We don't know if other people do, but we're going to talk about our growth a little bit. So last time we were on the eighth page of Google and we had made 12 cents. This week we were at the end of the second page. And when Rushali checked Hot New Tech on Google we were the first result on the third page, which is interesting. And we do want to make an SEO episode one time to figure out how all that stuff is working, but basically a second or third page on Google, as opposed to eighth last week. And money-wise, we still have only made 12 cents with web monetization, but we've added a Patreon. Actually, Rushali added it, so thank you, Rushali. Patreon is a service where people can give us money and we have two patrons now. So we now are making officially $30 a month, but after fees, I think it's $25.

Rushali (22:32): We should mention who our patrons are because we have to do shout outs to them. We should give shout outs to your patron

Judith (22:42):My patron is my mom, who is my biggest supporter. Thank you, mom. Yes. She also has offered to transcribe the episodes. She's doing such an amazing job and I can't wait to put them up. She's doing it with a lot of love and expertise. So I'm very excited about that.

So, okay. $30 from Patrion. And lastly, last week we had a Store episode and we put up a PayPal offering of physical coloring books and a cryptocurrency offering of digital coloring books. No one has given us any Bitcoin, which is very sad. However, we've made $75 USD selling coloring books, which are not actually in existence. Yet. Rushali and I have been working on them. I think we've fleshed out that it will be "The A through Z of Computer Science Fundamentals". And then we fleshed out all the words that we're going to do. And we've started to flesh out the actual designs and we have a goal of launching by November. Mid to end of November. Let's just say end of 2020. You know, under promise and over deliver. But anyway, so I think the total is around a hundred dollars, a little bit more, no, a hundred dollars, a hundred dollars and 12 cents I think is our total this week.

So in summary, we are now on the second or third page of Google and have made a hundred dollars.