Getting music to load through CDO's media.

DragonFireGames · 2024-04-04T22:26:42+00:00

I'm pretty sure playSound() uses https://studio.code.org/media?u= to load and cache its music. As such, I am trying to load music through it and it won't wor...

DragonFireGames

Letti42
I wonder what happens if you do a different format like this:
https://example.com/path/to/music.wav?param=song.mp3

DragonFireGames

That should mean you could also get data like this though:
https://example.com/path/to/music.mp3?query=data&a=a.mp3
Though, in my case, just using URL params is shorter & nicer

Letti42

DragonFireGames i thought about that too, but I've never had to use query parameters too much. does it work? have you tried it?

Varrience

Letti42 it should work.... i mean my yt and tts basically use it already but I'd have to recommission them cuz replit and i just haven't cared enough to put them back online.... well besides the yt-music thing

DragonFireGames

Letti42 Just tried it, it works

I used this on the server

app.get("/test", async (req, res) => {
  console.log(req.query);
  var data = await fetch("https://dl.sndup.net/dbs8/electromedieval.mp3");
  var blob = await data.blob();
  res.set("Content-Type", blob.type);
  var arrbuf = await blob.arrayBuffer();
  var buf = Buffer.from(arrbuf);
  res.send(buf);
});

Then I ran this in the console and heard a sound.
playSound("https://your.server/test?data=test&u=a.mp3")

Checking the logs I found:
{ data: 'test', u: 'a.mp3' }

DragonFireGames

Varrience Kinda off topic, but have you tried chunking the data for the yt-video thing?

DragonFireGames

Since I don't know if the bottleneck is in the actual fetching of the mp4 data, splicing that into frames, or loading those frames in code.org. If it's the middle one, it might be faster to splice it in chunks and load those video chunks at a delay. If it's the latter, then you can just send the frames in chunks. You could also try doing that for the audio as well, not sure how well it'd work though.

Varrience

DragonFireGames ffmpeg runs like dogwater in general that chunking may be faster but you'd have to wait more in between even slicing mp3's into chunks takes a bit.... you'd have to come up with some chunking system that allows for fast separation and rejoining which then requires subsections to be managed within the data, so you'd have to manage that and if your not caching it will definitely be longer than just me rendering videos like I've shown before

ffmpeg is good if you have specs though don't get me wrong
maybe with the right server that would be possible
and sure you probably could separate the data by yourself but i think it's pretty minimal though you could try though it's going to require a lot more memory management though

DragonFireGames

Varrience Well, I'm assuming that you are not caching it. Perhaps slicing a video into 5 second segments and then cutting up the frames would be faster. At least you could start watching the video. The idea is that while you watch a segment, the next one is being cut up and sent.

Varrience

DragonFireGames maybe so.... you could take a shot at it, i was pretty satisfied with my music player the video player requires much more space and storage and you'll probably be able to run like 1 video with active chunking unless you plan on making a powerful server to handle it, that's not the only thing though CDO has an image cache limit of which i don't remember the exact number but it will fail after it has too many images.... either that or referencing CDO storage loadins made it crash idk which but that's something to keep in mind as well

not only that there tends to be some failure rate with transferring the data as well and could possibly overwhelm the host or CDO depending on how your going to store it as well

DragonFireGames

Varrience Well, for my gif loader, I actually stack the images on top of one-another into a spritesheet and then turn that into the builtin cdo animations

window.loadGIF = function(url,speed,callback,callback2,failure) {
  var serverurl = server+"/gif?url="+url;
  var gif = {};
  gif.width = 1;
  gif.height = 1;
  gif.draw = function() {};
  loadImage(url,function(frame) {
    gif.width = frame.width;
    gif.height = frame.height;
    gif.draw = function(x,y,width,height) {
      image(frame,x,y,width,height);
    };
    loadImage(serverurl, function (sheetImage) {
      var spriteSheet = loadSpriteSheet(sheetImage, frame.width, frame.height, sheetImage.height/frame.height);
      gif.animation = loadAnimation(spriteSheet);
      gif.animation.offX = frame.width/2;
      gif.animation.offY = frame.height/2;
      gif.animation.frameDelay = speed || 4;
      gif.draw = function(x,y,width,height) {
        translate(x,y);
        scale(width/this.width,height/this.height);
        this.animation.draw();
        scale(this.width/width,this.height/height);
        translate(-x,-y);
      };
      if (typeof callback == 'function')  callback(gif);
    }, failure);
    if (typeof callback2 == 'function')  callback2();
  },failure);
  return gif;
};

Server

var gm = require("gm");
var request = require("request");
app.get("/gif", async (req, res) => {
  let url = req.query.url;
  if (!url) return;
  console.log("Starting: " + url);
  gm(request(url))
    .coalesce()
    .append()
    .toBuffer((err, buffer) => {
      if (err) {
        throw err;
      } else {
        console.log("Done with: " + url);
        res.set("Content-Type", "image/png");
        res.send(buffer);
        //fs.writeFile("/cache/"+encodeURIComponent(url)+".png",buffer)
      }
    });
});

Varrience

DragonFireGames hmm that might work..... idk how much frames you can run into the animation thing though I've never really played around with it much so it might be possible given that you could run a gif with an mp3 in sync well besides the animation maker for my emulator obviously you'd basically be making a pseudo m3u8 file

DragonFireGames

Varrience I would presume it is a decent amount of frames. Hopefully enough for a 5 second clip. Either way, I'm too busy with Codemon at the moment, but getting a youtube player working in cdo would be awesome.

Varrience

DragonFireGames technically I'm a bit busy too tbh I've been assembling something to solve acks question figured out i don't need to encode the data to load it in through b64 i can do it through the canvas instead

DragonFireGames

Varrience ah, that's cool

Letti42

i had an old project that streamed live sports on code.org, i used ffmpeg to extract the frames and audio - the best quality where it could maintain a constant stream was at 5 fps at 320x180 (default width of an applab project)

DragonFireGames

Letti42 code pls, thats pretty cool

« Previous Page