posted
So, I have recovered the source, post disk failure (yay), and can now resume development on release 7.
I should probably read the above for a list of things I was working on, but I think I prefer to just sand down the rough edges I encounter when using it.
It feels a little broken at the moment, like signals are much quieter on the new computer (I think they are, long cables), but I think maybe some hard-core testing could lead to significant improvements in note detection. I think the filters are in generally better shape than ever, but the peak detection algorithms have drifted into instability.
But I have a lot of quirky UI at the moment, and should probably try to get that under control.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
OK, everything is rebalanced now, though I can't see avoiding a volume control much longer.
That being said, I feel pretty good about the tone-detection again. Simple 'recorder duets' scan pretty well, though it clearly has trouble keeping up with fast notes.
I read some more stuff on FM synthesis and realized my implementation was actually something else.
I made mine more 'book perfect' and now I get the sort of sounds I expected. Very DX7-ish. Cheaper computation than I was doing, as well.
In theory, I support nine 'operators' in a flexible algorithm layout, but in fact all my tablets stall out around six, when you play four notes at once.
I must have 1000x the CPU of an original DX7, yet I am at best a pale imitation. This actually sounds pretty good though. I haven't done much playing around with it yet, but it seems to have a lot more interesting regions than before. Cascading modulators now actually does something that sounds sensible.
My limit of 64 'levels' (in this case, 64 frequency variations in the modulator) is probably excessively limiting.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
Sigh... I ordered another Kindle Fire! Clearly addicted. There is a special on the 10" version (about $100) and that's a pretty nice tablet, if you don't mind the advertising.
And now I will have to play every level of Tower Raiders Gold again! (something I am compelled to do on every tablet)
I definitely have to make a new FM synth video. I'm really liking the synSpace FM synth sounds.
----
Just to blabber on more about FM Synth. The general Idea is to have a standard 'operator' (an oscillator with output gain control, an envelope generator, and the ability to be frequency modulated). In my case, each of my nine oscillators can act as one of these 'operators' (though I only have six envelope generators)
Each operator is either a 'carrier' (in which case its output is sent to 'the speaker'), or a 'modulator' (in which case its output is not heard directly, but instead used to modulate the frequency of a carrier)
In my case, while all nine oscillators can be set to carrier or modulator, the modulator is always the oscillator immediately to the left of the carrier.
A Yamaha DX7 has 32 pre-defined 'algorithms' which combine six operators in various combinations and stacks of modulators/carriers.
if all your operators are set to be carriers, then no modulation occurs and you have a simple additive synth, adding the sine wave outputs of the carriers. (each operator can be assigned a frequency offset, in musical half-steps) from the 'current note frequency'
In the simple case of only two operators (just mute the ones you don't want to use), changing the relative frequency offsets of modulator and carrier will have a strong effect on the harmonic content of the carrier output. Adjusting the output level of the modulator intensifies the effect. Using an envelope to control the modulator level allows you to shape the harmonic content of the note.
Using an envelope to control the carrier output level just controls loudness, as normal.
Modulators also have a 'feedback' value which just injects some regenerative feedback in the oscillator, which boils down to changing the output to something other than a sine wave. At max feedback, it approximates a sawtooth wave.
But, of course, my oscillators already have real sawtooth outputs, if you like, so it's all very redundant.
Still, these FM sawtooths are not 'pure' and, overall, the FM 'sound' is pretty musical, no matter what you do. Which is why it is satisfying to play with.
Adding additional modulators to the stack is, for me, a bit harder to predict the effects of (chaos, nirvana, or nothing. take your pick!), but I imagine you can stumble upon some magical settings.
And it's all set on a per oscillator basis, so you can mix and match additive, subtractive, and FM synthesis. Not that it adds value to be able to do that, just that you can :-)
~5min additive synth example ~15min subtractive synth example ~18min FM synth example ~30min FM Feedback example
You probably want to skip immediately to the 30min mark if you've been following along. Hopefully this completes my synthesis digressions, though my wife has challenged me to make a Theremin.
---
Next I think I go back to work on my chord detector with the world's cheesiest neural net.
posted
I finally decided to make my chord detector a standalone class since I think I might need three of them at the same time, as it were, listening to the microphone, what's being played on the keyboard, and the contents of a pre-recorded groove.
So each user creates a Chord object (really a ChordManager object where I plan to stick all sorts of useful chord functions) and then feeds its stream into it by creating a series of histogram objects.
So there is a bit of API for building a histogram
code:
chord.clear(minEnergyRequired) for each note being played 'right now' energy = loudnessOfNote chord.setEnergy( noteId, loudnessOfNote) chord.submit()
You probably do that at a regular interval, reporting the current active notes at each interval.
the submit call (names subject to change and likely never seen by the player, so I am just blabbering) indicates the histogram is complete, so it is then analyzed. First it gets normalized since the energy units are local to the user and Chord has no idea, but it keeps track of min/max values seen and the user provides a min energy value below which the note is ignored.
It also folds the histogram onto a single octave, so there are only 12 buckets in the histogram, and all "C"s go into the same bucket, no matter the octave. I add the energies
Then some magic happens and the normalized eenergy histogram is turned into a 'chordId' (like C Maj7)
This chordId is available to the user if they just want to display 'the current chord', but it is also added to a queue, with timestamps (timestamps are also unitless and can vary with the user).
This allows the user to more easily scan data, say if you want to render a little moving ribbon of the most recent chordIds seen.
code:
int numChords = chord.numChords( startTime, endTime) int chordId = chord.nthChord( i ) int timeStamp = chord.nthTimeStamp( i )
So when you call numChords, you provide a region of time you are interested in, and it returns the number of chords in the queue in that range, plus it sets some points so the calls to nthXXX() don't have to repeat any searching (but you lose the context almost immediately)
And then some support routines for the actual rendering of a chordId in text, or maybe guitar chord image.
Anyway, if I do it right, all the actual chord stuff can be in one class, and the users will only have to come up with 10 or 20 lines of code of their own.
----
Of course, I haven't done 'the magic' part yet. Again, I hope to do a very simple neural net for that eventually, if only as a programming exercise, but I could also do a completely mechanical deconstruction based on the loudest notes in the histogram.
And then, finally, if I actually am successful tracking chords by name, hopefully I can use that info to properly guess what KEY SIGNATURE a song is being performed in.
I might have to watch all those YouTube videos again by Michael New (hope I got that right) to get my chord foo up to speed.
The trick is mainly knowing which energies are bogus (harmonics of other notes). But in theory I handle that at the layer below this (before I build the histograms). I think this layer will just sort of assume that it is being given good energy data.
I will think in terms of triads, with 7th and 11th extensions. Because of inversions and my folding the octaves, I can't depend on the root of the chord being the lowest note.
But assuming I have a sorted lists of notes being played, and the number of halfsteps between them all, I think I could just start on each note in turn, and see if I can make a legal chord out of it
code:
for each note root = note; // we start with a guess n = numHalfStepsToNextNote( note ) if ( n < 3 ) { dissonance, this is not the root } if( n == 3 ) { check if rest of notes fit in minor key } if( n == 4) { check if rest of notes fit in major key } if( n == 7 ) { perfect fifth, look for displaced middle }
any mismatches make it bail on the suggested root and move on to the next starting note (when going to 'next' note, it wraps around as needed)
ANYWAY, that's the plan, something like that. I think I most want to see this stuff on the Vocoder rhythm rainbow, but also on the sequencer piano roll.
Oh, I should point out that the chord history only advances when the chord changes. (and holds stable for a moment).
Plus, that history holds all I need, I think, to estimate the keysignature during a particular time stamp range.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
tonight I pursued various objectives and had to wave off of each of them in turn for one digressive reason or another, but finally landed on the fact that my percussion hits were not aligned with my synth hits.
This is because I cheated and used sampled instruments for the percussion. So instead of going through my normal synth engine to generate samples and stream them out at the proper moments, I just ask the OS "play this sound effect file" (basically)
I could tell something was wrong, but my wife had to point it out to me. There is a known problem in android where sound effects played in this way can have a lot of lag in their playback, and it's not something I have any control over (and probably varies from device to device), so I just assumed this lag was the cause of the misalignment in rhythm
until finally I noticed the sampled sound was actually playing a bit BEFORE the synth sound, not after it.
And, of course, that makes sense. I have to sort of preload my sounds, and they must be buffered on the far side of my 'stream buffer chain' so when I trigger them, they actually play faster than when I add samples to the output stream, which then has to get resampled before it gets to the same place the recording starts.
So, lord forgive me, I added a fixed delay and hand tuned it to sync the hits (125ms).
Now there is still a lot of 'jitter' in when the sampled waveform starts, so it sounds like a not very good drummer, but at least it is errors 'centered on the beat' instead of always to the same side, as it were.
I think there is some brain wiring here, picking something that sounds 'right'. Des the speed of sound vary with the freq of the tone? Would you hear the high freqs sooner? In which case, I should have the crash lead the pad, as it were. But it was leading before by about 150ms and that sounded bad
anyway, sounds more connected now (percussion and synth rhythm), I think. Though I'm sure I will regret the hand tuned value, since it probably needs to vary with the device, but I am doing it as a nnumber of samples and my known sampling rate, so there is SOME science behind it.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
worked on the chord detector engine some more. this is fun :-) nothing to show for that yet since I got digressed into 'harmonic removal'
When I find a spectral peak, I need to know if that is an actual note, or just a harmonic of some other note. If it's just a harmonic, then I need to ignore it when it comes to extracting the melody of the song (but keep it when it comes to the tonality)
I started a little neural net for that just before my last trip to the hospital :-) It wasn't working yet, of course, but I think it's probably a good idea in the long run, especially if it can learn in real time by listening for the harmonic mix of specific instruments.
ANYWAY, I needed to remove these harmonics TODAY, so I wrote a simple scrubber which actually seems to work.
Starting with my spectrum measurement (an array of energy values, one for every possible MIDI note, with the energies I am hearing 'right now')
I pass that into my peak detector, which now is just looking at the first derivative and finding inflection points to find the peaks. But this finds all peaks, including harmonics.
Then I scan that from the left (lowest notes) and for each peak detected, I scan the notes to the right of that peak, and if I find a note with energy, which is an harmonic of the note I am checking, I diminish its energy by a percentage of the energy of the note I am checking. If the energy is diminished (attenuated) enough, then I kill that peak -- declaring it was just a harmonic
But if an actual second note were playing, that matches that same harmonic, my attenuation, hopefully, leaves a little energy left in the bucket and so the peak is not removed.
Then I resume with my normal 'ok, I have some peaks, now look for note starts and ends' which tracks the energy of individual notes over time. This guy is a little bouncy at the moment and likely to add notes.
RESULTS
probably will add yet another video to show this. I think it's a step forward, but still not perfect. I have some more manually tweaked constants in this section.
Overall, it cuts down significantly on error notes, but not 100% and any errors are, of course, immediately confusing. Plus it tends to filter out quiet notes. So you get extra notes from the loud parts, and missing notes in the quiet spots.
Musically, I reiterate, this feature is really just to let you capture melodies you sing or hum into the game and then you use the track editor to actually turn that into your composition. And for that, it's probably already good enough, but every time I improve it a little I know there is so much more that could be better. Just have to stop when it's stable and not muck with it for awhile :-)
oh, it's zillion times better in picking out bass lines now, that is very satisfying.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
So, I'm going to declare success on harmonic removal and now I am back to note detection.
And I noticed that certain notes seemed to get filtered out. For example, when using this really cool test data of a recorder playing a simple song. The G and A notes are substantially quieter than the C note, even though to the human it sounds fine (and the musician is probably doing it on purpose even, if it isn't just a side effect of recorder physics)
Anyway, it turned out that since the recorder puts out pretty uch pure sine waves, that apparently, from my perspective, G &A are just oh so slightly out of tune, and the Q of my filters was so hugh that if yu were just slightly off, then no energy appeared at all.
So by LOWERing the Q, my note filters now accept a wider range around each note and can sense these 'slightly off tune sine waves.'
More complex sounds have less of a problem as they create more noise around the center freq and the filter sees that as more energetic somehow than an offcenter sine wave. Or so it appears.
more hand-tuned values...
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
OK, got my first cut chord detector working. The actual breakdown of the chord was pretty easy once the incoming data was better. I will definitely make a video of this, sorry!
Just to blabber about it...
My magic cable delivers clear audio of sufficient volume to my vocoder. This is key.
My spectral recorder does a reasonable, but not perfect, job of analysing the change in energy over time for each note of the MIDI keyboard.
To analyze a single spectrum, I go first through a first derivative to find the points of inflection and then establish the 'peaks' within that spectrum.
Then for each peak, I determine if it is a real musical note, or just a harmonic of a lower real note. I suppress these harmonics (hiding their otherwise legitimate peaks)
Then, for each remaining peak, I try to determine the moment it first started, and when it eventually drops below some level. These give me note start and end moments. This part still needs more work, but has improved recently.
So, now, while I am making a live recording, these peaks are used to determine the set of all notes which were playing during each individual spectrum (24 spectrums per second, but should probably be faster)
That's like an 8 octave range or so, but I 'fold' it onto itself, so I have a single octave of data, where the first bucket holds the sum of all the peak energies which were 'the note C' somewhere on the keyboard. The second bucket has the sum of all the C#s, etc.
I then turn that into a bitmap (11 bit number, each bit representing one of the 12 semitones)
If we call that number a 'chordId' then I decode it into a text description like this:
code:
// I don't even know the root of the chord, so I // try all possible roots, then see if the other // notes in the chordId indicate major/minor/7thness. // this routine lets me pull the value of a single bit/note
boolean isNotePresent( int chordId, int offset ) { return 0 != chordId & (1 << (offset % 12)); }
The mod 12 allows us to supply large offsets, which are then folded down to their appropriate single octave offset.
So the actual decoding looks like this:
code:
int root; // will get 0-11, the root note of the chord for( root=0; root<12; root++) { // we try each possible root in turn // extract the booleans for which notes // are present. boolean hasRoot = isNotePresent( chordId, root ) boolean hasFifth = isNotePresent( chordId, root + 7) boolean hasMajorThird = isNotePresent( chordId, root+4) boolean hasMinorThird = isNotePresent( chordId, root+3) boolean hasSeventh = isNotePresent(chordId, root+11) }
Anyway, now I know which chordish notes are present, for each root. I think start to demand things.
For now, I declare a chord to need 3 or more notes. The root, the fifth, and either a major or a minor 3rd. Without those features, I assume this root is not correct and move on.
But if it meets the triad pattern, then I just do some simple logic to decorate the root letter. So I turn that into text like "C" "Cm" and "C7"
I tweaked my rhythm rainbow (currently shown only during recording) to show that 11 tone folded scale in real time, with ROYGBIV rainbow colors stretched over the 12 tone range (red = C)
So that scrolls along showing you the last 10 seconds or so of chord info, including the text name of the chord.
And in one of those rare moments, this algorithm turns out to be all you need. It's only concerned that the required notes be there, and is not upset by 'extra' notes hanging around (and I tend to have lots of those), unless they make it look like a reasonable, but different, chord, and my error notes tend to be 'out of the current scale' so they don't meet the pattern and don't mess things up.
If I wanted to handle more than major/minor/seventh chords, then I would probably be more fragile, but this feels pretty robust right now.
Again, given a clear signal and a good recording.
There are a bunch of youtube videos with people teaching chord progressions, and those are fun to play through the system.
----
Now, right now, I only detect actual chords where you hold the notes at the same time. But if I fold time, as well as frequency, and sort of merge the chordIds of the last few frames, I should be able to detect 'arpeggiated' chords as well as 'held' chords.
Anyway, it was a good weekend.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
Once you've seen the chord names scroll by, you've pretty much seen it all.
----
Cool, I got blocked on YouTube! And it looks like it was a completely automated check. And they offer a feature which is supposed to edit out the song but leave the video otherwise undamaged, so I am trying that.
Not completely sure why the original video hosting the song was not flagged, but hey, I like to do the right thing!
Also, I don't see a button to actually take down/delete any of my videos. Do we not have the right to do that?
posted
So, there are a zillion free patches out there for FM synthesis, and I thought "wouldn't it be nice to include a representative set of instruments"
the free-to-use patches are all in sysex format, which is gobbldy gook I don't feel like learning today.
So, instead I installed "dexed" (free FM synth for PC) which comes with ten or twenty thousand of these unrestricted patches (i.e. NOT the Yamaha patches!)
So you can load up an instrument in dexed and see the six operators and their settings. Then you can try to convert that to synspace equivalents.
note: third harmonic (3x freq) is base + 19 half steps, which is also "perfect fifth plus an octave"
The very first thing I realized is that my contour generators are lacking a pretty fundamental aspect, which means I can't really imitate the typical ADSR stuff. My 'D' as it were always plays to completion BEFORE the 'R' starts, So if you lift the key in the middle of decay, it still plays any remaining 'green' area of the CG before fading out with the red Release area.
Which I can't believe I never noticed. It's not a show stopper, but it's pretty newb.
That aside, I still failed because, honestly, dexed is just that much better than my stuff. If you really want to get serious about FM synth on a PC, you should try dexed. I believe it works as a complete MIDI player as well.
So, I am going to use this as an excuse to digress into improving my FM synth. Which might be as simple as allowing 'cent' deviations in all the frequencies.
But might also be due to my 'only 64 levels allowed' attenuators.... though, since I have a separate CG per operator, I guess my transitions are higher rez than that.
Or, quite likely, my 16KHz sampling freq.
But I'd like to be sure my math is right at those points, at least.
----
having a 'keyboard split' is starting to make sense to me. and I definitely suffer from the lack of 'cents' Maybe not so much from the attenuators
----
Well, my fourth try sounds somewhat piano-like, but more honky tonk than the one I was trying to imitate.
What was slightly more effective was to play the instrument in dexed with various operators turned off, so as to see the effect of one oscillator at a time and then do the same thing in synSpace and tune it from the inside out.
My lack of 'cents' means I end up with different beat frequencies (larger, in general) than my example patch, and that is pretty significant musically. I will have to deal with this (UI and data changes).
my lack of splitting the keyboard gives me settings that overdrive at higher freqs, which I just have to avoid or tune against. But I just don't see myself adding support for this. Maybe do it automatically.. or with a single checkbox.
And the rest of the difference is the 16KHz sampling, I think. I just can't do a lot of math above 8KHz so the output is missing the higher harmonics. But I can work around this by working with lower tones (and try to get an exact match there, and hope it scales well to the top of my range).
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
I added a 'detune' setting to each of the nine oscillators. It only appears if you enable the detune checkbox on the KNOBs page.
Thereafter, you see the current detune value, in 'cents' positive or negative. Tapping around the number changes it.
There are 1200 cents in an octave, distributed logarithmically so each has the same amount of perceived pitch change. There are 100 cents per semitone (halftone).
I believe these are the units used by the DX7, who limited the range to +/- 7, I think. I am limited to +/- 30 cents.
In the simplest case, this allows you to make nice chorus effects, with 'backup singer oscillators' playing slightly sharp and flat of 'you'.
Perhaps someday I will add a 'delay' factor, that delays individual oscillators in a fixed or time-varying amount.
Having precise control over FM Modulator frequencies is a big step forward, and I think I can continue to defer worrying about the 16KHz sample rate.
I still need to fix my contour generators... (jump immediately to release on key lift)
In the process of making a handful of instrument patches (pretty nice piano and hammond organ), I wrote down 10 or 20 bugs I have been experiencing for a long time. I vow to quash those this weekend.
Oh right, the reason for posting... My complicated perfect cabling solution.. was perfect, but that one cable (LINE-to-mic level conversion) was very noisy (I would hear static as soon as I plugged it in)
Well, guess what, if I unplug the kindle USB, there is no noise. To get noise I need both the kindle USB and the converter cable plugged in at the same time.
I'm going to wave my hand and say 'ground loop' and not worry about it. It's scary clean (no noise) when I disconnect the USB. Of course, then I lose charging.
Oh, right, I added a OPTION/NERD to mute the live output during recording. With my magic cable set up, I found any live output during recording would ultmately loop back around leading to sticky notes that never end. Which is in itself freaky cool in small doses.
Otherwise, you can listen to the live output only with headphones plugged in.
There is no compelling need to listen to live output :-) Someday, if I implement 'beat matching' (changing my beats per minute value to match what I am hearing, and slowing to get myself in sync). Once I have that, then I can predictively play future notes, instead of always just reacting to what I just heard.
If I am just reacting, it's like singing into an echo chamber and very difficult to continue unless completely in step,
But if the machine has worked out "oh, he's arppegiating a G chord" it can join in with something harmonically appropriate, and be 'on the beat' (better than me, presumably).
Your personal backup singer whose only goal is to make you sound good. or better, at least.
---
I also continue to want some sort of Song Designer starmap that lets you map out chord changes on a simple 16 bar (or whatever) song sheet. Using generic chord names I II III... VI VII applied to the currently selected key signature
Then some sort of auto-arpeggiation stuff and it (the script) shoves the song down into the sequencer, which can then play it, or edit/save/clone it using the normal tools.
posted
OK, as promised (well, ONE of my promises came true), I fixed a zillion little bugs I had been stepping over for awhile now.
* cloning an editted GROOVE now actually saves the modified notes! (oops!)
* TRACK EDITOR now has a nice 'play cursor' that shows notes both before and after being played
* TRACK EDITOR now has a nice zoom +/- control to show note events with different scale factors
* KEYSIG control on GROOVE panel now can step through root keys in either order, and separately change the mode (major/minor). You're still stuck in 4/4 time, however, but if I ever support 3/4, this is how you will set it.
* BPM meter no longer lags out for large songs
* max events per track increased (AGAIN), still not enough.
* player ship radar had been a little broken for awhile now, all better now.
* fixed my glaring omission in the Controur Generators. Now an 'early release' causes a rapid (linear) movement 'to the official release point' and then plays the original release as designed.
* fixed reverb such that turning in on and off no longer causes a huge change in sound levels.
* added/improved "Chord Rainbow" control, with live chord determinations (during vocoder recording and playback). If you know your rainbow colors: ROYGBIV, those map to notes CDEFGAB and sharps/flats are a brighter/dimmer color of their 'root' so C is 'red' and c-sharp is 'light red', for example. You learn the chords by color surprisingly quickly.
* the button that turns the oscilloscope off and on, now shows the selected scope channel name (like MIX for the final output, or OSC_3 for the output of oscillator 3).
Clearly, I am starting to feel like my old self. About time.
I also just 'cleaned my room' in a long overdue way. Moved my old pooter off to the hand-me-down pile, so now I have a little more room for my feet, and a few fewer cables snaking all over the place.
This takes me to the task of turning my Oculus SDK2 back on after six months of no VR. And once I have recovered that ability, I need to try VORPX and see if it really is a way to get an old DX9 game into VR without a full rewrite.
----
Begin blabbering.
I suspect the future is standalone VR/AR goggles, where the 'phone' inside is doing 'all' the work and driving the display, playing the game, etc.
But in the mean time, we still have 'VR on PC, that can sometimes be fed into a cheap phoneVR, but usually requires an expensive PC VR.
I don't ever expect to be the sort of guy that someone buys a VR set-up in order to try out my stuff. I see myself more as the guy who has some stuff you might as well try, given that you already got the VR.
So, to that end, I don't mind that I only work with an $800 PC VR, since the point is you already had that.
But I would still prefer to work with a cheaper standalone unit, I'm just not sure if that is a good time investment (PCs are dead, right? :-)
The problem is that my old code is old enough that it needs a lot of secondary upgrades (new compilers, new APIs, etc) before it can use 'real' VR, and I want a magic fix that lets me at least approximate things with a lot less work.
Enter VORPX.
After I restore my oculus setup, I will buy VORPX and just try it as a user. The documentation is pretty weak, but what it LOOKs like is that that interpose themselves in the directX protocol stack. So my program issues the old boring monocular commands it always did, but they look at them first and, I claim, change the arguments sent to the graphics card, probably sending alternate frames to left/right eyes.
In theory, you could retrofit a lot of games just by overriding the camera transforms. Though I imagine a frame rate hit/halving which could be an issue. Also, could they do their own head position adjustments... I suspect not without real hints from the game, and not sure how the game..l.well, again. I am just going to try it.
And, finally, Steam. I promised to look into that this weekend, and I haven't yet.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
More puttering and room cleaning in advance of 'really getting something done'
* I now track edits to your current Patch and put up a little (modified) marker. You can still screw yourself by accidentally tapping a shortcut without saving your changes first, but now that I have the 'dirty bit', I can eventually provide some protection for that. (haven't decided if I would rather have a popup are-you-sure, or if I can cover it with some sort of UNDO option)
* found (not fixed) a bug that if you start the game and immediately record a GROOVE without first picking a PATCH (i.e. just using whatever patch it booted up with), then it won't record the patch id properly in the groove (will show as unknown) and you will have to assign a patch later via the BAND page.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
Didn't feel up to working, nor even playing NES Classic, so I made a video that more thoroughly documents the vocoder in version 1.07 (to be considered 'the documentation', if a user manual were in a mumbled monotone...
This one is pretty long, just to warn you, but it describes each of those goofy controls inside the vocoder.
And this time I am using Premier to do the conversion to something more upload-friendly (still a huge file) so I have no excuse for doing absolutely no editing. Probably could have boosted the audio in just a few clicks. I suck. Sorry!
I can't believe how long it's taking to convert this 2.3GB file! It's been over 30 seconds already! (seriously, I am so used to things happening instantly on this new pooter, that it really is a shock when something requires real time).
Holy Cow! Premiere messed up the audio sync! Or rather I managed to do that by using Premiere.
Hmm, this is also no smaller than the original, so I think I will just upload the original and make a note to learn Premiere at some future time.
posted
Heh, I continue to make radical improvements to the vocoder without actually the result of it sounding any better :-)
I went through a round of cleanup of recently-added things and some performance improvements (I love my 10" Kindle, but it is dangerous to develop with it since it is so much faster than other tablets and gives you a false sense of security)
I've also gotten a bit more experience with the Android Studio Profiler stuff, which is pretty excellent, though has not yet pointed me to anything slow that I could actually fix.
I usually profile in the less-accurate-but-faster sampled mode, which generally complains about my large number of canvas drawLine calls. But when I use the more invasive inline profiling, it doesn't really mention drawLines and points to a different set of functions. So I'm not sure if I am chasing false clues or not.
My general speedup is to render anything complicated to an offscreen bitmap, only when it changes, and then render that bitmap to the screen every frame. But that turns out, in many cases, t o be slower, at which point I have to ask "are all these lines working together to make something beautiful, or just messy?" since the real way to speed something up is just to not do it at all.
And, in particular, not show everything at the same time, done in some beautiful ergonomic way that expands effortlessly into relevant details from a functional home screen.
Following the android 'activity' metaphor is probably sound advice. Break your app into focussed pieces that relate to individually useful activities.
And, I guess, by implication, the only navigation button is 'back' (where I would traditionally have a bunch of 'tab' buttons to switch directly between peer displays instead of 'out to main' and then back in)
Plus I want it to draw my attention to anything I am doing that leads to unnecessary thread waiting.
...
Yes. well. It's not news that I should edit more.
Here's an annoyance. SynSpace:Drone Runners can import starmaps from your Google Drive. They must be "Text Files" and the path of least resistance creates "Google Docs" which are not the same thing. So you need a 'text editor plug in for google drive' and I have been using "DRIVE NOTEPAD" since I started.
But not lately. I just tried today and apparently DRIVE NOTEPAD has been broken and unsupported for awhile now, with lots of sad users. Sad users, like me, who never paid a dime to use it. I probably even groused about it. My ingratitude no doubt contributed to its apparent demise.
I am using a replacement now, but haven't really commited to it because... it's.. different. It also has obnoxious advertising, but I should be willing to accept that. Except the ad eats into some pretty valuable real estate, all the time. But maybe it's a true notepad (intended for small grocery lists), and I am really looking for a code editor that works directly on text.
I mainly depend on bookmarks and searching and 'got used to' the weird way DRIVE NOTEPAD handles that. (and now I am also learning how Android Studio does that as well)
ANYWAY, when you first run into it, it feels like all your files are lost to you, but you just have to open them with a different editor. Google Drive hasn't lost them, and they are still text files.
Various people have proposed various stump and dead cat solutions to get DRIVE NOTEPAD to work this year, but none of their prescriptions worked for me, and it sounds like a pretty fundamental api change from google needs to be adapted to by DRIVE plugins.
----
I tweaked the colors in my 'chord rainbow' and am expanding to using them in a couple of other places (but not in the TRACK EDITOR where that much color would explode your head). When looking at a compressed piano roll scroll by (especially when folded into a single octave like the chord rainbow control) really benefits from the color spectrum and can actually give you some quick insights "song uses 'C' (red) a lot"
the mapping is notes C D E F G A B .. C
map to rainbow ROYGBIV red orange yellow green blue indigo violet
And the sharps/flats show up as blended variations on those colors
----
I was thinking it might be fun to render the FACE icon of your currently selected target in the middle of the TRIGGER control. And make sure a starmap can declare the sound effect/synth sound to play when the trigger is pressed. Though I like the core UI to be very light weight and 'hollow'
----
* I re-did the social text control (that mini text keyboard) to render to an offscreen bitmap instead of rendering a bunch of rectangles and text every frame. No apparent improvement in performance, and slightly inferior graphically. Not a satisfying reward for some quirky coding.
* I added two more buttons to the main vocoder control buttons, filling the remain gaps which I depended upon to easily find the button I wanted.
* The PLAYBACK MODE button (I bet I already documented this, sorry) is to the lef t of the PLAY button and controls what happens when you tap PLAY.
- SPEC will playback a spectral recording - FMNT will playback as a formant recording - SAMP will playback the raw recorded samples
Spectral and Formant recordings are memory limited to about 10,000 frames (at 25 frames per second) (400ish seconds.. six-plus minutes). But raw samples are only available for the most recent minute of recording. (asking for samples outside that range will just loop into the existing samples)
Just tap the button to step through the modes, and it might take a second for the change to take effect.
* The microphone preamp control is just to the right of the RECORD button (the vocoder record button) Tapping the TOP HALF increases the preamp and tapping the BOTTOM HALF decreases it. It's a very small control
Each tap raises or lowers the preAmp by one decibel, for a range of -9dB to +9dB, and defaults to +3dB since that was best for me.
Ideally, you would adjust the volume of your sound source to match what your device can record well, but this lets you adjust to under- or over- driven signals.
* I changed the spectrum display so the white bars (note detected) are now using the rainbow colors... again, when things are popping, makes it easy to see 'oh, that was an F!'
* In theory I made some major improvments to 'note detection' in both time and pitch. But still too far to go to be worth bragging about. Filter computation remains my number one cpu usage. And the funny thing is you know the device probably has the same/better filter math baked into its firmware/hardware :-)
----
Oh, you can also adjust the 'vocoder top half display mode' by tapping in the left qurter of the top half of the display.. In theory the display modes are the same as the playback modes, but only SPEC and FMNT render stuff. SAMP just renders as SPEC, I think.
slighly less mysterious than before, and the xxxx/yyyy counter in the top left will be the anchor for that UI (tap here to change display mode)
----
I'm so glad I caught the missing feature in my contour generators. Piano-like envelopes are much more satisfying now.
----
YAMAHA DX7 BLABBERING
I know I have gone on and on and on about FM synthesis and how much I loved the sound of the Yamaha DX7 synth. And how excited I am that my fm synth implementation manages to make similar sounds.
I just wanted to point out that I honor all yamaha copyrights (and heap great praise upon them). As far as particular synthesizer settings go, I am not seeking specific numeric settings for Yamaha stock DX7 patches. That would feel wrong to me, though I'm not sure where the law actually stands on that. it would just feel wrong.
However, a million musicians rolled their own patches and published them for free re-use, and I feel OK about plumbing that data.
As it turns out, seeing the raw data numbers is not a huge help since my fm synth implementation uses entirely different knobs with different range settings. But there is a good enough correlation, that starting with 'known to be good DX7 values' I can usually get close pretty quickly.
But this process has been working for me to make my own family of instruments inspired by work real musicians did for the DX7
* first get DEXED, a synth standalone/plug-in that implements FM synth pretty much exactly like the DX7 does.. I don't know if that has some sort of math-copyright issue or not. I hope not, but accept it is not my problem to solve.
* the same links that locate DEXED also have libraries of open-license patches in a file format that DEXED can load (multiple blocks of 32 instruments each, per file)
* That all installed easily (nothing fancier than ZIP, I think) and DEXED sounds wonderful, and if all you were doing was looking for an FM synth module for your DAW, then you can stop now.
But if your goal is to make an instrument PATCH for synSpace (and why wouldn't it be?) then do this:
* pick a dexed patch you want to try to replicate. Take youre time. You will be spending quite awhile with this, and if you don't love it now, you're not going to love it more later.
* note the algorithm and how the six fm 'operators' are wired up. You will need to assign (up to) six oscillators in synSpace to perform the same roles. And since synSpace has a horizontal perspective, you won't be able to use the same indices and will either have to keep track in your head or make yourself a little cheat sheet. But job one is to set the synSpace algorithm values in the oscillators until you achieve the same modulator/carrier setup as the instrument you want to copy * next I like to go through and set the frequency of all the oscillators, and make sure they are set to sine wave and distortion is off, etc. I generally map each oscillator to its own contour generator, but often the dexed data would support sharing CGs in many cases. * I will usually try to make a very very rough match to the envelopes, especially for the carriers.
Once I have set all my own knobs to what I think are reasonably similar settings as to the dexed knobs (which I just look at and interpret -- there is no page of numbers to look at anywhere in this process), I start tuning it by ear.
For this, I pick a convenient octave (middle C ish, say) on both synspace and dexed, so I can easily play the exact same note on each (alternately or simultaneously). In fact, I am in my two-earbud mode, so I hear dexed in my right ear, and synSpace in my left. (and I can use the vocoder as an oscilloscope for dexed if I need to)
then I disable all but one carrier (in both dexed and synspace), and disable all the modulators that drive that carrier.
Then I compare dexed to synspace. This should just be a sine wave at this point, but one where you can get the envelope to match for the carrier. Also, the pitch should be identical unless you made a mistake, so it's good to find that now.
repeat that for each of the carriers individually, and also all enabled together (but still with no modulators) and make sure you get a similar (boring) sound from both dexed and synSpace
Then go back to a single carrier enabled only, and add in exactly one of its modulators. Then tweak synSpace knobs until those sound as close as possible (at no time do you tweak dexed knobs in this process -- that would make a moving target!)
Basically, if you set the freq properly, then only bits to tune are the level (the dB value in synSpace) and the envelope assigned.
I think I could make myself a tool that would make it easier to imitate the dexed envelope (basic ADSR), but right now I am just sort of using a handful of basic shapes and look at dexed only enough to say 'oh, it's a long slow one with sustain' and pick my standard one of those, as it were. Then tweak it more later if it sounds too wrong.
But I find it's not too hard to find the same setting of dB and CG to imitate the contribution of a single modulator to a single carrier, and if you do that individually for all carriers and modulators, you end up with something fairly similar to your original, and then you can hand tune it from there to your liking.
synSpace does not currently support the feature that lets you blend you values a bit depending on which octave you are in. So, for example, a 'feedback' value that sounds great in the middle of the keyboard, probably goes into screeching white noise if you play notes in the highest octave. A real DX7/dexed can scale down the feedback value as you pick higher notes.
But synSpace does offer more per-oscillator tuning choices than before:
* FREQ OFFSET (in half-steps: +12 is one octave up). Most FM works in scale factors, like "2x" for an octave. To help you work that out, I display the factor equivalent for whatever offset you have selected (offset +19 for factor 3x for example)
* DETUNE (in semitone 'cents' +/- 30 cents from perfect pitch) Again, this is per oscillator
* ABS/RELATIVE. If you pick ABS, then all the normal math is done, but relative to 440Hz instead of the current note. The actual freq that turns into is displayed (instead of the scale factor)
----- Anyway, I think that's the last few weekends. I just sat down to pre-test Google Drive on my new pooter in advance of making a movie this weekend about adding a new starmap, and ran into the DRIVE NOTEPAD snafu (which I really hope resolves itself, but it appears to have been broken for months now, so that's not a great sign)
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
So, rather than do any of the logical things I ought to do in order to get 1.07 released, I thought of something else I crave to add.
Basically an arpeggiated chord player, coupled with a sort of song layout editor, so you can easily define a song as a series of chord progressions, organized with intro, outtro, verse, chorus, coda, etc.
You would first set your desired key signature with my funky control for that. Then a new keyboard would appear (much evolution in design there) with seven buttons labelled with roman numerals I - VII
tapping one of those would select the Ith chord of the current key. Only one chord can be selected at a time, and the key probably lights up. Might have two modes (lights only when held, or toggles on tap)
Whenever there is a selected chord, its triad is played in arpeggiated form, over and over, until you turn it off or select a different chord.
At first I thought I should leverage the existing loops to do the arpeggiation, but the bottom line is that when working in 'key' space, as opposed to 'scale' space, you need to think in terms of the seven proper notes of the key, as opposed to the full 12 notes in our spectrum.
So the design I am circling works like this:
* There is the existing piano keyboard with 2.5 octaves of keys (black and white)
* There would be a button somewhere to take the piano in and out of 'chord mode'
* When in chord mode, the left side of the keyboard (one octave's worth of buttons) is turned into the chord engine. (which is rendered basically on top of those piano keys)
(and maybe someday I will have drum mode with a special hexpad area you can bind to a subset of the percussion instruments)
But you can think of the chording engine as two rows of seven buttons. The bottom row is the 'push to play chord I-VII' buttons.
The top row is a bunch of modifiers which can be held BEFORE you touch the 'chord I-VII' buttons, and modifies them.
So, for example, there are modifiers that affect which notes are in the chord, like MINOR, 7tH, AUGMENTED, etc. I probably only will offer 2 or 3 mods for this. And the MINOR modifier would probably mean 'flip it from its natural state into the opposite' since in the KEY SIG of C Major, the 7 individual chords are some major and some minor.
Then other modifiers are used to set up the arpeggiation. And since this is a very simple pattern and generally quite short, rather than link it to a loop index, I think I will maintain a special memory blob for the 'current appegiation' and to set the pattern you would
* touch and hold the arpeggiation modifier * tap the I-VII keys in the order you want the arpeggiation to play them. In this context the I-VII keys represent placeholders for the actual notes.. But normally an arpegiation would be something like I,III,V for a 'root, third, fifth' upwards cycle of three notes (repeated if you keep the chord active).
At first I thought the arpeggiator would need to worry about major/minor etc, but really that is baked into the whole I II III... business, where if the KEY is CMajor, then the chords rooted on each of the 7 tones in that key are already hardwired to major and minor. So the TRIAD (root, third, fifth) is completely controlled by the key signature and the selected I-VII chord. The arpegiator normally only plays the notes of the triad (or octave offsets of those notes), but when I add the modifier '7TH' it adds a note (and come to think of it, how do I factor that into the arpeggiation?)
So, still some gray areas as my brain gets stretched to appreciate the differences between keys, scales, and chords.
But at least I think I have reduced the UI to something small enough to fit without looking like too much of a nightmare.
and the song composition, I repeat, really belongs in a starmap script, not baked into the engine. But the engine being able to play a complex chord stream from a small amount of event info in the groove seems useful. Sort of a 'chording track'
I think I would actually create a new 'midi event' for this 'chord change' which would be an omnibus packet including the current:
* chordId (or none, if silent) * arpeggiator arguments * current bpm * current keysig * current timesig
Then I would spit out one of these at the start and end of every song, and then presses on the chord keyboard would add chord change values throughout the song.
In theory they could appear in the track editor as something you could see and modify.
But at some point I need to be able to 'express' them into lots of individual notes in a track of their own, so after making a simple chord progression song that you like, you would convert it to 'all events' and then you could edit the individual notes.
I'm thinking there would be another modifier for 'speed' which again you would hold that then tap I-VII to set to the speed to 1-7. Where 1 would mean 1 arpeggiation note per beat (slow). 3 would be 'triplets' etc.
I dunno about 'note duration' and whether or not they should be allowed to overlap. Apparently, traditionally, arpeggiation was 'invented' specificaly for monophonic instruments (e.g. flute) which cannot play three notes of a chord all at once, and hence depend on them being played rapidly enough for the audience to fuse them into a chord. So that would vote for 'no overlap between arp notes'
This will also be a good opportunity to see if I can do a clean class definition to make this somewhat reusable.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
still enjoying the 'design' phase of the chording arpeggiator, drawing it over and over again in my quadrille workbook..
There is this dichotomy between the '12 notes in an octave' vs 'the seven notes in a scale' For a chording keyboard, this pulls me between a representation where you pick the root each time, and add some modifiers for minor, suspended, etc. This requires at least 12 buttons plus modifiers
Versus a representation where you separately pick a key signature and then I only offer the seven chords of that key signature. And no real mods, other than to the arpegiation pattern, if any. This requires only seven buttons (plus key sig selection) which makes it favorable. Also, it sort of confines you into 'things that sound nice'
But it confines you. It limits your free choice. But if you wanted free choice, you would just play the chords yourself and be done with it.
So I think the keySignature-based solution is better here, and the primary labels on the buttons will be the roman numerals, but probably a secondary label showing the actual root for current keysig.
And I will probably color the buttons with their proper ROYGBIV colors (so the colors will move as you select a different keysig... might be odd.. or educational).
So, I will still have the two rows of buttons, but the labels on the second row will change (and a prompt might be added) when you are in the middle of some change. Like if you want to change the speed of the arpeggiation, you will tap the speed button, then the I-VII buttons will relabel to look like speed selections, and you will tap the one you like (or tap the SPEED button a second time to keep the current setting and return to normal I-VII buttons.)
for the arpegiator itself, I still see the need for:
* a mode. basically: - no arpeggiation, just a full triad - triad with strummed start/end - full arpeggiation, as recorded.
* a speed, how long each note of arp lasts - 16th note - triplet 8 (3 notes in a single eighth note) - 8th note - triplet 4 (3 notes in a single quarter note) - quarter note
I don't need a 'num notes' any more, since I am going to just record the arpeggiation as it is 'played' once on the I-VII keys.
It will be assumed to loop constantly over the life of the I-VII button being held (but you can lift early to abort the arp at any time individual notes play to completion if started)
-
Mainly this weekend is about fixing an ongoing issue at work that I have gotten zero traction on. So far, the only synSpace changes I have made have been in the Track Editor and the piano shortcut buttons, for cosmetic effect.
HOWEVER, I did add one new feature to the track editor. Its along the left hand side of the detailed track view (full screen track editor), and it's something to allow you to move ALL the notes of one track, a little to the left or right.
To use it, you set the START cursor to 'the first note of your song' (depending on how long it took you to start playing after pressing RECORD, this might be some distance into your GROOVE recording).
Then set the PLAY cursor to the location you WISH is where the first note of your song was.
Then LONG-PRESS this new button. It will then beep and slide things over as necessary to achieve your goal.
I tested it like this:
* I made four vocoder recordings of the same song (with minor vocoder mic boost tweaks between them)
* I saved each to a separate track of a 'new groove'
I then played them all back at the same time. Since they were NOT synched up, this led to a rather cool syncopated sound. Sort of a dance mix of the song
Then I added this feature, and was able to re-align the four tracks so they all started at the same time, and any differences were recording differences (say you wanted to evaluate what worked better for vocoder sampling)
This also sounds cool, though not as cool as the syncopation did.
In theory, I could then use the track editor to manually edit ONE of the tracks to contain only the very best captured notes from all four tracks, to get a perfect song in memory, then delete the other tracks. And, of course, your goal is to create, not imitate.
---
So, as I struggle to learn how to compose music, I am constantly creating 'rules' which must not be broken, lest it 'sound bad'. Obviously, there are no rules and you must follow your... muse. But I do believe in a lot of music theory as to what sounds nice, what leads the human to expect/desire a certain resolution, what is surprising/exciting, what the rhythm is telling you, etc.
And some emotions are triggered specifically by a change in key signature. And for that to work you have to clearly dip the audience in the first key long enough and effectively enough, that they are prepared to hear the transition to the new key, which you must then hold long enough for them to appreciate the change and yearn for a return to the original key... before they have forgotten all about it.
And the transition can be made less jarring if you sort of sneak into it. One way to sneak is by using a chord which exists in both keys. Each of the 12 basic chords, exists in six of the 12 basic keys (and is not used in the other six keys), with the exact mix looking chaotic, but actually making sense if you have a 'circle of fifths' to use as a 'ruler'
In any case, so I am thinking, and I probably tried to explain this previously, after you play a chord... you get extra buttons for the five other keys where that chord is 'legal' (and an indication of the I-VII status of the current chord in that destination keysignature). So If I have some chord progression I want to end on, I have to get myself to the start of that progression at the proper moment to then just play out the final progression in the final keysignature.
And I have the choice of jumping into that new key, using, say, a perfect fifth (V) in the final key, setting myself up for a pleasant conclusion on the root (I)
But my jump to the new key is from the old, and the jump interval there can be different from the jump interval measured by the new key, so I might want to be gentle during a transition, to coax the listener into admitting that each new note could be justified either by a note before or a note to come, and any tensions were short lived and non-cacaphonous.
ANYWAY, with this little circle-of-fifths derived helper control, as you randomly chorded your way around you might notice "hey, the chord I am playing right now in the key of G is also the V chord of the key of C, so my next chord change could legally be any interval supported in G, or any pleasant-enough interval supported in C" (adjust preceding to get a key pair where it makes sense)
But I probably want to 'strum/arpeggiate' my way across the boundary, to soften the blow, and use chord inversions to keep the overall octave consistent and mellow. Inversions are probably to be done in the track editor (which might benefit from a shortcut to slide selection up/down one octave)
---
I try to keep asking myself how I, a neophyte, feel this will help me noodle actual songs and symphonies I don't mind sticking on starmaps.
In theory, the track editor is 'good enough' that I could create anything my heart desires, but it would still be tedious (can't see enough of the song, or see too much, no sense of overall structure, no easy re-use of phrases, or repetitions in general, and a general lack of talent). So I need the tool to address those and make me look good.
Probably I need to store some number of fragments, where a fragment is a rhythmic chord (or note) progression of no more than, say, four measures or so.
Then be able to see important time stamps in the ruler, with intelligent zooming, paste these fragments into place.
A pasted fragment, is still a fragment (a reference to a model somewhere else) but has its own transforms, so it can be shifted in pitch and time, including the override for individual notes while otherwise retaining the fragment (so it inherits changes to the model fragment)
And then a sort of madlib questionaire where you enter three phrases and four key signatures with BPM changes. And edit those in real time and changes are applied instantly, as the song, or the begin/end marker portion thereof, plays over and over.
And THAT should be driven by a lua script. (with API support for collecting input from piano keyboard and lua-huds)
But there's always an underlying engine-support requirement for these lua APIs and the trick is to put just the right stuff on the engine side to let lua be super powerful, but also super lazy and avoid realtime lua issues (though lua is really very fast!)
(New) Things I am OK with the engine doing:
* understanding the nature of a song being repeated phrases, with variations and callbacks, and lua should be able to define these phrases in a little code book of some size, but that code book itself is held in the engine for use at 'runtime'
* Say you start with four madlib values (intro, verse, chorus, outro) Then you provide a short string that shows the order used in the song. Each fragment you provide is as long as you like (but think 4 measures anyway)
IVCVCVCVCO
Then a button somewhere generates a track full of note events to achieve the playing of those fragments in that order. Pretty robotically, one imagines.
So if I could use my loop engine as my fragment storage... probably not. But a GROOVE would want to keep a fragment pile around.
You can edit (track editor) the generated masterpiece to make it more better and stuff, but if you push the magic 'express' button, it will regenerate (well, add a new track to the groove based on the current fragment definitions and 'song layout string')
Also should add some additional markers for coda, bridge, alternate paths, etc. So you can make a little state machine that plays as long as you like with enough variety. (this being music for a game after all). Grooves should have the opportunity to repeat endlessly, in fact I have omitted that.
Actually, I think there are reasons to restart a long song over and over (I mean play it again sequentially) instead of baking in an infinite coda in the song definition string. I think it's important that the song definition string define something of finite length, however long that is.
And I drift into this because I want the chording arpeggiator to be the thin end of the wedge into the 'song structure editor with fragment library stored in groove'
You know... I had a CASIO toy once back in the day that allowed simple monphonic note entry where you would tap the notes one at a time, and it would record them without any timing (all just quarter notes played one after the other), but you could take your time and make sure you hit the right notes without performance anxiety. And maybe there was even a delete button.
ANYWAY, *after* you entered your notes, you hit this OTHER key repeatedly with the actual rhythm you wanted. Not sure if it allowed notes of varying length or not. I just bring it up because of the separation of rhythm, which is actually a nice concept.
The Song Structure Definition String thing feels oddly empowering. It's enough to decorate the track editor timeline ruler, and tell a co-player what measure of the song they are in, if they are, say, adding another track playing along with this one.
I should definitely leverage the existing track stuff, so adding additional tracks is easy and you can choose when to delete old ones you don't like, or clone the whole set for later.
But where can I store fragments, and not be redundant (shared with the arpeggiation for sure, )
I don't want the fragments to be full-on event streams, just simple short arrays of note gate descriptions (start time, duration, pitch, stress) (leave the instrument definition outside the fragment? remember it as a default?)
So that makes this a different category of data.. I will try to coerce to six bits of info per note for efficient serialization
So if this is to be part of the chording arpegiator then the I-VII buttons at some point are used to store up to seven fragments, and you can select any one of them to be the current arpegiator, and they are remembered with the GROOVE
But perhaps they are known as fragments I through VII and you develop your own feeling for which roman numeral means "chorus" And the song description string now looks like "I ( II III II III IV II III II III IV )2 V"
Where (...)2 means 'do ... twice'
I don't really like roman numerals here, let's use ABCDEFG
Though those are also to be confused with note values... but..
"C (A B A B D A B A B D)2 E"
Where A is verse B is chorus (ok, I'll use C probably) C is intro D is bridge E is outro (...)2 section is played twice
Yes, something like that. I like that. Could actually use the note colors in displays where it wouldn't confuse the issue. With two more letters available, (we have seven!) we could have two variations on the verse, to bake in some variety in the song structure description string.
Then, after that is robotically expressed into a track full of notes, you manually track edit it to perfection..
yes.. I still like it, but how to enter/edit the fragments. Should really be able to use the track editor to edit a fragment. And fragments need to live in '12' space, I think. So they should be entered with the full piano keyboard, woth the chording arpeggiator overlay sort of minimized, but blinking to get out of the way while you are actively entering a fragment. You either have to enter the fragment with proper timing (performance anxiety, but the editor is right there, and fragments should be short) (don't forget that fragments can include chords!)
So that I don't like so much, sounds complicated and confusing. But at one level, I should probably cut in and make 'virtual tracks' for the fragments. So you record one like any other track, then open the chording arpeggiator and push some button that means "save this track as fragment A-G"
Now that I like a lot. You have full recording editing possibilities (could even use vocoder) to make the track, then I have a single button in the UI, and a well-defined task "read the track and turn it into a fragment note list suitable for storage in one of the seven fragment storage bins which are serialized and saved as part of the GROOVE, making it just that much bigger.
A fragment note list probably has several kinds of events in it, each with a timestamp, the whole thing sorted and ordered by timestamp and event type. So look ma, I broke my promise. All of a sudden the note list looks like a full on event list, but no, I will turn this into a sequence of simple letters! I will!
fragments are stored in a relative fashion, with some initial constants at the front of the fragment locking them down in time and space. After that are TIME markers indicatig the timestamp for all subsequent events until the next one that needs a new tmiestamp (which will be frequent) time moves in 16th notes at the current beats per minute setting.
Something like Ff -- declare start of fragment, f is base note Tn -- advance time by n sixteenth notes Bn -- begin note at +n halfsteps from base note En -- end note at +n halfsteps from base note
So a simple arpegiation might appear as
"F0B0T4E0B4T4E4B7T4E7"
That would be three notes in a row, each lasting 1/4 note in length (T4 is 4 sixteenths notes)
Now, you would never type this (unless up in your lua code there was some reason), it would just be how I serialize the data when storing it inside the GROOVE object. I don't mind 2 characters per event. The second character (in my system) can hold 64 values (meaning any 64 things you like, say -32 to +31, or 0 to 63, or 0 to 1024 in steps of 16, or some state id value..). To the degree is is human readable, that's always nice, but not a requirement.
You would always record your fragment as if it were in the key of C, based on a C key on the mini keyboard, and you would be forced to stay within some distance of that note for the life of the fragment (though you can replay the fragment later with offsets in time and space... is that true? Not in a way that doesn't require modifying the song structure string... Well, in any case, that's an issue for the structure string and doesn't have to impact here other than "you only have a 64 note range. Hmm... what am I saying... it should just use the 5 octave minia piano range and hard wire the base note as needed for that. Yes, no translation needed and fits inside of six bits. true? I offer five octaves, plus C.C#.D.D# so yeah, exactly 64. d'oh. So I don't need the 'F' marker in the beginning
Now, instead of End note markers, I could include duration in the Begin note marker. For a live recording, I oftern don't know this, but here I have a completed track recording to work from. so I guess that would become
Ds - Note duration is 's' sixteenth notes until told otherwise Nf - start a note at piano pitch f Ts - advance time by 's' sixteenth notes
Now, will I regret all those "Ts' when it turns out there is one in front of every N record. If true, then maybe the N record would have a T value baked in that was a built in T command (advanced T before starting the note) So the T value would be 'this event happens 't' sixteenth notes after the previous event'
Dd - set note durations to 'd' steps Nft - advance time 't' steps and then start note f (note will end 'd' steps later)
Which is a little better, but '3' is so ugly, so why not bite the bullet and just have the N record
Nftd - advance clock by T, start note at pitch f, end note d steps later
we could agree that pitch 0 (or pitch 63) meant 'silence' in case we needed a longer 't' value than 64 sixteenth notes and could emit 'silent note' to advance the clock. Or, we could have another event (justifying our use of N)
Tnnn - jump time 'nnn' steps (up to 999 sixteenth notes)... Well, let;s say we wanted to be able to jump a minute at 60 beats per minute in 4/4 time, so that would be 240 sixteenth notes, or about 8 bits of data. too much for my sixbit encapsulation, so I would need at least 2 characters for the argument.
I doon't think a FRAGMENT should have BPM or key changes inside of it (though it might well represent those in the song structure string)
A fragment assumes:
* an instrument has been assigned to play it * it has been transposed to a new key possibly * event times are relative to time fragment starts
---
OK, where am I in my blabber?
* I feel good about ability to create phrases from track recordings (selections within track recordings, in fact).
* I feel good about storing them in seven buckets inside the GROOVE
* I feel good about expressing a phrase into a new track full of notes, as directed by a Song Structure String.
* I feel good about the song structure string being able to define a repeating pop song of some complexity with minimal input.
OK.. OK.. so.. where does the chording arpeggiator fit into that? That's all about notes played manually on the piano (or encoded via vocoder). Where does my sitting there pressing chord buttons at periodic intervals come in?
I think they come in as part of a track recording to be used as a phrase. If I just play keyboard notes, they get added to the track as individual notes. If I turn on and use the chording arpeggiator, then each time I switch to a new chord, it starts repeating the arpegiation for that chord and emits note events as if I were playing well, and those events get recorded to the track.
So... in that scenario the 'chord progression' is never stored in some high level form. You get to push the buttons in a high level way, but they are instantly converted into a cloud of notes.
Yeah, I am really hoping for a 'document' that I guess allows me to define a phrase as a chord progression....
hmmmm...
If I have a T and N, why not a C
Cvka - start chord 'v' in the current key - with the current arpeggiation
And this is aligned with how I thought I would have the engine support for the arpeggiator itself.
So... I would 'record a track normally using all the existing track recording opportunities' but add the ability to turn chording arpeggiator I-VII buttons into Chord Start events in the track (much as I embed Loop Start events already)
Then those would be as editable as I care to make them, but ultimately I could play back a track that only had these chord events in it (no direct tapping of piano notes, just chording arpeggiator taps) and they would be 'just in time' expressed back into the notes required per the current arpregiator settings, and no 'notes' would be stored in the track at all, making the GROOVE smaller amongst other things.
THEN, when I push the button to 'save current track selection as one of seven phrases' it is smart enough to turn these chord events into full on note events (with arpeggiation, if present) in the new track.
So... implemenation maximal joy comes from
* implement the lowest level of the arpegiator engine, controlled by some fixed constants, just to make sure I can smoothly generate what I want, and store it in a groove track, along with normal note and loop events. Probably add support for some keysig and BPM events
* blocking out the UI (moving piano between modes of chording arpeggiator, normal piano, (and someday drumkit maybe). I am currently blocked on this, worrying about the button to select this option. Worst case the existing piano button could toggle between several piano display states.
* making some pretty buttons for the chording arpegiator and lay out the full UI for all features planned. Make sure it is thumb-friendly on a phone.
* implement handlers for the pretty buttons, focussing first on the ones that set Arpegiator settings, and store phrases.
* wire the chord buttons to emit 'midi chord events' to the GROOVE
* wire the live chord buttons to arpeggiate with current settings
* extend that for GROOVE playback
* tweak out the arpegiation settings
* tweak out the phrase creation process
* provide some phrase preview functionality
* tweak out the arpeggiator engine and lua-fy it nicely
Can I literally use my Chat Text keyboard to collect a manually-typed song structure string? Why yes, that would be completely OK with me. Then an enterprising starmap can focus on building that string with a nice UI.
So, where is the button that applies the song structure string? I guess I will make it like the clone button where you start with the existing value... hmmm.. then again, I'd hate to have to make a real editor there, with the ability to set the insert cursor. Yeah, I would have to trick that out and be sure there was room for a long enough string
Nothing is ever completely simple. I will keep trying to reduce this, but I think this describes the total functionality desired.
* play some musical phrases into a track * select them one by one in Track editor and save as phrases in Chording Arpeggiator * build a song structure string, using A-G to mean the seven phrases, and (...)n as a repeat n times marker, and maybe other fancies like changing BPM and keysig, * phrases can contain both note and arpeggiated chord events
Getting there... have to back up and make sure it's fun.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
so, more revisionism. The pitch detector (array of filters) has been working pretty well for awhile now, but I noticed some notes seemed 'cleaner' than others.
As I'm sure I mentioned, I wasn't sure if I was getting a noisy signal from my 'online tone generator' or maybe picking up some resonance in my magic cable from PC to tablet.
I was making another demo video last night and ran into this effect again, so I actually did something good and added a test case that basically overwrites what comes in from the microphone with 'mathematically perfect sine waves that step over the entire spectrum'
And sure enough, even with mathematically pure sine waves, there was an unexpected variability in the per note filter response, with several notes barely showing any peak at all over their silent neighbors.
I messed around in various ways until I found that something I added a few months ago, which I praised as a giant leap forward (using an exact number of full wavelengths when feeding the filter) was, indeed, making this bad thing happen.
And when I removed that optimization, my filters started treating the 'pure' data as expected and all my filter responses look the same from C2 to C7, and much sharper as well. Too sharp, I decided, and lowered my 'Q' values, which also required a lot of re-tuning.
But my point is, having real, known, test data, and removing sources of noise and distraction, is the very best way to actually make progress, and it was only 5 minutes of coding.
So the pitch detector is yet again radically improved, and this time, it might actually be improved enough that someone other than me might notice :-)
I re-sampled all my standard test songs and they ALL got a better version than before. Especially in the quiet spots.
Another thing I changed was my definition of 'energy'. Before, I would run some number of samples through the filter and then compute the sum of the squares of all the values that came out. Then I qould sqrt that sum and divide it by the number of samples, as I understood this would give me the RMS energy.
But now I suspect I just made that up, and the squaring was distorting the energy measurement I cared about. So now I am just summing the absVal of the values (getting the area 'under the curve' An integration as it were) and then dividing that by the number of samples. I had to rebalance a bunch of stuff after that (of course), but I *think* that was an improvement.
I took about 2 hours of video in several films and plan to actually EDIT THEM down to something useful with the most boring bits removed. We'll see how that turns out.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
So, I've already moved on and broken things again :-) To recap recent changes for 1.07
* Vocoder has a 'display mode' toggle, so you can visualize your spectral recording as:
- a bunch of spectrums edge-on (neat freq space view) - a bunch of 'formant harmonics' (fun for analyzing voice) - raw sample blocks (energy content of groups of samples).
* Vocoder has a separate 'playback mode' toggle that controls what you hear when you press play
- a reconstruction of the original sine waves recorded (spectral playback). Sounds like a loud concert heard through a thick brick wall. - a simulation of a human vocal tract, sort of. An excitation oscillator (currently a sawtooth, buzzing at the lowest pitch detected in the recording) fed through a parallel array of filters, whose outputs are merely summed and sent on to the mixer for you to enjoy. This makes a sort of robot voice. - the actual raw samples. Only a minute's worth of that is available (if your recording is longer than that, you will hear the same samples repeat, but the final minute of the recording should have synced samples). This is worse sounding than it should be, at present.
* if you tap in the current spectral display, you move the 'current spectrum' cursor to that point (and immediately begin hearing its recreation). This is in addition to manipulating the 'spectrum range' control.
There is definitely some 'play' value in just making little recordings and looking at them in various ways.
Anyway, I repaired the 'unequal filter response' bug and saw a nightmarish improvement in note detection. I could have just stopped there, probably.
But there were dropout spots still, where my test recordings (recorder music, next step down from mathematically pure sine waves) had some notes with dramatically lower energy than the ones immediately before or after.
My note detector favors the loudest events and these notes simply looked to weak. My attempt to address that was basically to lower the trigger threshhold, which works, but also lets in a bunch of other stuff that legitimately was just noise.
This led to some improvements in visualization and playback until I could see it wasn't my imagination, but the note coming from the recorder was *profoundly* different for these couple of notes I was regularly missing.
And when I critically reviewed those notes, it was clear that the 'problem' is that those notes had loud harmonics (2-5) while the clean notes had only the lightest second harmonic. The human ear was hearing all of those, and the total energy of that was not all that different from the 'pure' tones nearby.
But mister one-freq-at-a-time computer was only listening to the fundamentals, so to him it sounded like a 4x drop in energy.
People with no lives will remember that I actively suppress harmonics. I do this by:
* scan all freqs for energy into a spectrogram * identify the 'peaks' in that spectrogram * for each peak, look to see if there are also peaks at frequencies which would be harmonic to this one. (multiples of the freq) * If I find any, I 'diminish' them. If they get sufficiently diminished, they stop being peaks * Then another pass looks at the remaining peaks, and thinks about how they change over time. This is a bit of 'AI' that ultimately needs to decide: - is it a note - make sure you detect start and end times well - from pitch, amplitude, and width, decide which 'stream' a note belongs to (percussion, rhythm, melody, harmony) and then emit a 'midi' event for the appropriate instrument.
I haven't done that last bit yet (multiple instruments from a single vocoding), but I suspect I will work on that next. To get started on that, I decided I would use my eight 'shortcut' keys (selecting my favorite instruments) as part of this, so that whatever you had bound to shortcut#1 is ALSO your 'preferred bass instrument', #2 is your preferred rhythm instrument, #3 is melody, #4 is harmonic (and #8 is percussion for arcane reasons that might change)
I think this works well, as it only recently dawned on me that the shortcuts represented a pre-selected group of instruments that you enjoyed switching between. They might as well be 'a band' which then does its best to collaboratively perform the vocoding.
---
Anyway, back to note detection and harmonic suppression. So here was this F#/G coming from some baritone recorder (or something) that was just full of harmonic content. My harmonic suppressor was doing a good job of suppressing those peaks (so well that even spectral playback had the note sound weakened)
But all that energy was just being wasted.
So... I turned my harmonic suppressor into a vampire! If I think you are a harmonic of 'this note', then not only do I suck energy out of YOU, but I inject that energy into this note! Thus restoring it to the same effective loudness as I might get from one of its less harmonic fellows.
Anyway, that actually worked, but I still have the lower threshhold as well, so now it is note city arizona, and my vocoded grooves are just a solid wall of notes. When you play it back, it actually sounds kinda good.
And a lot of it is 'real' (actual sound placed there by the engineers who produced the piece)
But I think at least half of it is cymbal crashes being turned into 15 note 'chords' of no particular quality. Most of that comes from the higher octaves, and this is a task I think comes under the heading of 'ai' (in the lowest way) where the exact same note detection might be handled differently (sent to a different stream) based on the events leading up to it.
If there appears to be a bass line, then a detection of C3 might be a 'high note' of the bass, or a 'low note' of the melody. I might do this by just having a line that separates them, and then moving that line when notes 'get close' to it, and hope that bass and melody don't both explore the edge at the same time. I will probably start with a line that doesn't move (get the streams working), then move 'away from close notes, and then back to starting point when possible'
And maybe I only have to be 'smart' about percussion and bass/rhythm, and then 'everything else must be melody/harmony'
But I think I will want to characterize by note duration to some degree. A bass line is mostly going to be notes of same duration, I think. Whereas a melody is likely to have lots of variety there. So if that C3 has 'the current bass note duration estimate' then it goes to the bass stream.
And eventually you end up with a set of features you can detect, and then if you want to get all neural netty, you have some good data to drive things with, and the net is probably pretty small.
I'm not thinking net at this time, just algorithms.
But hey, the high quality vocoding, short lived as it might have been, was pretty cool. The actual purpose of the vocoder (cool synthetic inter-player alien speech) was also slightly undercut as the filter improvements brought the recovered voice dangerously close to being understandable. But adding the formant playback mode knocked that back into safe incomprehensibility.
I suspect I have gotten close to re-inventing the Speak 'n Spell. It sounds a bit like that. And if you tap on individual parts of the spectral recording, you can play back individual vowels and such, and they are VERY speak and spelly.
----
I think I broadened my understanding of voice though, which I thought I already knew everything about (yeah, I think that a lot).
On day one of signal processing class, you learn that any signal can be broken down into a weighted sum of sine waves, and I really took that to heart.
I knew that feeding a harmonically rich oscillation into a resonant cavity, was different from 'adding together sine waves of all the related frequencies', but still, I always run back to 'adding sine waves' so it was a bit of a gumption trap to actually do it with filters instead, but I'm glad I finally did.
Note: this is not the same as my formant oscillators, where I was going to, once again, add together sine waves. Where the magic would be that the oscilators were optimized to glide through the transitions seen in formant recordings of speech.
Instead, my pipe dream now is to have 'formant filters' which do similar gliding of their center frequencies, but the sound actually comes from a single oscillator (sawtooth, rich in all harmonics, buzzing at your 'singing' pitch. When you talk, this is around 50-200Hz, when you sing it goes up to 2K ish
That oscillator is passed through each of the formant filters, and what comes out is 'mostly a sine wave at the formant freq' and in theory, I am tracking the filter center to follow the formant pitch, so I always get 'about the same portion' of the energy, no matter what the base pitch is.
Except at higher pitches, most harmonics are past the filter cutoffs, so you naturally get a more sine-wavy signal from high voices (apparently making it difficult to elocute clear vowels in opera.. arias?)
ANYWAY, so my formant player runs a sawtooth at the freq of the lowest peak of the spectrogram, and then sets up bandpass filters at each of the other peaks from the spectrogram (yes, up to 128 filters!) and whatever part of that sawtooth that makes it through, is what you hear.
And right now, you can still make out the buzz of that sawtooth (note: plan is to let you adjust that buzz pitch and filter centers, to get the alien voice of your dreams)
But it has also reached the level I need, where I can tell that you're talking, that you are some sort of sentient creature, but there are just enough buzz and clicks that I can only tell what you're saying if I already knew what it was (at which point it's a little too easy, so slowly pronounced profanities would probably make it through)
But aside from regressing into the 'wall of notes' again, I'm really pleased with the results of last weekend. And the importance of structured testing with good visualizations and test data.
And let me plug
YouCanPlayIt.com
(they have a youtube presence where you can play along on your recorder to their clear versions of various songs. For the vocoder, this means a source of clear, almost-sinusoidal-but-still-just-a-little-tainted-by-the-real-world tones.)
And my theory is that first my stuff must work with this 'clean' data, and then I introduce noisier stuff, while keeping it working with the clean data through regression tests.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
OK, my attempt to state something clearly, to prove to myself I understand it (and then fail).
Animal Voice Model
* vocal chords
are represented by a sawtooth oscillator, which 'buzzes' while producing 'voiced' sounds.
When speaking in a monotone, this pitch doesn't change much. When you shout or whisper, it varies the loudness of this oscillator.
Sentences ending with question marks raise this intonation on the final syllables.
Certain sub-texts are conveyed through word and phrase-length variations of pitch and loudness of the sawtooth.
Any playback of this spectrum requires both the filter coefficients and a description of this excitation oscillator at the moment of the recording.
* resonant cavity
The sawtooth is not heard directly (in most cases), but only after it has echoed around inside a resonant cavity including lungs, throat, sinuses, tongue, and lips. Humans can skillfully adjust this cavity in real time, monitoring the result, and hence convey a pretty good bandwidth of information. (faster than typing).
So a mathematically accurate simulation would have control sliders for 'tongue position' etc, and work out the exact resonances that would come out (probably limited to some cone of forward projection from the mouth, outside of which signal quality is not guaranteed)
Instead, I have an array of filters, each individually tuned, and one way or another, they are roughly tracking the center frequencies documented in the spectral recording (the peaks' intensities in the spectrum).
I accept this is not the same thing, but luckily I can run to my "it's not supposed to be comprehensible" defense.
ANYWAY, so when you talk, you are driving that excitation oscillator, and you are 'singing' all the time, though mostly just softly and off-key. If you imagine yourself just "humming what you're saying instead of saying it", that's mostly what your vocal chords/excitation oscillator are doing.
But we all know the kidnapped lady in the back of the car can clearly enough enunciate "Pizza" from behind the gag (name the movie!), so muddy as this signal might appear on its own, it carries great power and information content.
But this sawtooth is actually the sum of several sinewaves. The fundamental (your 'hum/sing' frequency), and it's natural harmonics at 2x, 3x, 4x, 5x, ... 15x your hum frequency. These are those 'fingers' you see in a formant playback.
Filters basically diminish some fingers, and not others, leaving you, in the end, with fewer fingers. So instead of hearing the raw sawtooth with all its scratchy sharp edges, you hear a mix of the lower harmonics, with the higher ones all but erased by filters which are emulating some position of the larynx (for example).
BTW, to ignore the importance of the airflow would also be a mistake. try humming with mouth closed and nostrils pinched. You need air to move. in or out.
to make 'vowels' you pretty much just have to alter the RELATIVE intensity of the first and second formants.
The trick here is 'what are the first and second formant frequencies? do you mean the fundamental and second harmonic (2x)? In which case, it would always be 2x and that would suck!
No, I think I mean (at this vary second) is that F1 and F2 are, indeed, just harmonics of the hum frequency... but not always the same harmonics.. varying with the controlled shape of the resonant cavity.
So when I see all these harmonics.. I know that ONE of them is F1 and ANOTHER is F2.
PROBABLY F1 is either the hum freq or 2x it. But F2 might be best chosen as just 'the loudest one of those remaining'
So, my natural hum frequency seems to be around 55 Hz, so my harmonics are 110, 220, 440, 880, 1760, 3520
I am going to rule out 3520
So, those are all octaves of musical note "A" which is nice. I have a personal key signature.
So if someone said my F2 was around 1600, I would say: I bet it was closer to 1760!
Since, when my vocal chords are hummming at 55Hz, the ONLY frequencies coming out of that are harmonics. there is no 1500 hz coming at all, just some 880 and 1760 by which you might infer a sense of 1500 ness, but the actual frequencies are limited to those which are simple harmonic multiples of my instantaneous hum frequency
Pretty much true, ignore the bits that aren't.
If I then increase to humming 57Hz, then the only freqs I am issuing are 57, 114, 228, 456, ... 1x 2x 3x etc.
And so the filters cannot create any other frequencies out of what I am giving them. They can just choose to selectively attenuate some harmonics and not others.
The resonant cavity, on the other hand, does not just move in tandem with the hum frequency. So I might hold my cavity constant, while varying hum, thus playing different sets of harmonic frequencies into the same set of filter tunings, resulting in a somewhat altered sound.
If I hum/sing in a higher pitch, then my hum freq harmonics are all much higher, and thus the higher harmonics are more likely to get filtered by most cavities. (they can also get amplified when the cavity is 'tuned' to match the harmonic in question.
So, to achieve a 'vowel' (which I can do at any reasonably low hum freq), I just need to
* make a suitable sawtooth at hum freq * make an array of filters that emphasize proper pair of formant frequencies (which must be harmonics of hum freq)
Done right, we hear the proper pair of formant harmonics, with the proper relative loudnesses, that we interpret as "ahhh ehhhh ihhhh ohhhh uhhhh" (and we can then modulate the hum frequency to get singing in vowels)
gliding smoothly between vowels is how we get dipthongs. There is no "eye" only "ah-ee"
breaks of silence/noise between vowels, are how we do consonants. Little puffs of white noise
And that thing we do with lips and teeth when saying "zzzz" That's some sort of "eh" vowel with a bit of airy white noise, just short of a whistle. Perhaps that is white noise (breath) passed through a bandpass high-Q at that whistle freq (set by tongue/tooth distance -- tongue-tip whistling, as opposed to lip-whistling, but both highQ filters resonatng at the whistlefreq, adjusted by mechanical control and live feedback)
I suspect my model would benefit from a whistle channel :-)
But my recent enlightenment is that while any signal can be made from the weighted sum of sinewaves (ok, ok, in general), at least for some finite length of time, it isn't that it is made up of completely random sine wave frequencies.
I will claim that had I paid attention on day TWO of signal processing class, I would be thinking more like "any PERIODIC signal, has a fundamental frequency, and could be described as the WEIGHTED SUM o f a set of sinewaves, who are at HARMONIC intervals to the fundamentl. And you should also use the proper phase for each of those harmonics, but I read somewhere humans don't really hear phase so much as phase change. So I always neglect to worry about the phase and just focus on the weighting.
Before, I was sort of thinking in the linear algebra sense where you could use any three non-colinear vectors to describe a vector basis (and this decomposition of signal into weighted sine waves seems similar to a vector base). But you get the most efficient/linear encoding if yout basis vectors are mutually orthogonal to the others (at right angles to), so that changes in the y value of a position do not also need to modify the x value.
So perhaps harmonics are just the equivalent to orthogonality in this space.
And maybe the repetitive nature of octaves in music is just a second harmonic effect, and there is an interesting repetitive nature of 'tri-octaves' for the harmonics.. so if 2x a freq is interesting because it is both an octave and the second harmonic, then maybe 3x a freq is also interesting, and 4x (well, I guess 2x burns all the even ones).. ok, looks like we're walking into prime numbers.
So, PRIME NUMBER harmonics (2, 3, 5, 7, 11, 13, 17, 19, 23, 27) should be the interesting ones (having no octave relations).
so back to 55Hz hum freq... the INTERESTING PRIME HARMONICS are 55 110 165, 275, 385, 605, 715 935.. so wow, getting to 2000 is quite the leap. and it looks like there will be 3-4 harmonics close enough to require scrutiny. Myabe do a sort of rolling average and try to find the centers of clusters of harmonics that made it through, then F2 is the center of the high cluster, and F1 is the center of the low cluster....
And if I do that from only looking at PRIME harmonics, then I should have to worry too much about "but did this come from a note, or just a harmonic of a lower note" Yeah... you know... this would imply that nine times out of ten, than F1 and F2 are NOT octaves of each other. (or that that is one very specific case, like "Ohhhhh" or something, and that for information production they will almost always have a non-octaval relationship) (except perhaps when singing arias?)
-------
You know, in earlier posts, I kept showing lists of harmonic frequencies, which were actually lists of octave frequencies, and left out lots of the ones I just enumerated, but they have always been there, I just think in octaves a lot, I guess).
This means there are more harmonics in general than I depicted before, and hence more are available to be attenuated/amplified by the filters, but again only the actual harmonic frequencies are available and do not change when attenutated.
Not sure if it affects my 'which harmonics should be considered F1 and F2 right this moment?' or not. more to decide between makes it harder :-)
------------
So, no, apparently I cannot be succinct, even when that was a stated goal.
But I like the bit about PRIME HARMONICS. Such an obvious thing! I now have an orthogonal vector basis with which to describe spectral content. Now I will google that and see that it's been all the rage for 400 years.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
Speaking of SpeakNSpell... I actually played a small part in that story. I was developing a product that was basically a 'home computer' just a little bit before we had 'home computers'. It used a 6502 and had a bunch of serial ports, so it could sit in the line between your Big computer, and some ascii console terminal. In that position it could chat with either you or your computer. I had made a simple language where you basically provided a test pattern (what to look for from particular serial ports) and what to do in response, which included typing commands, dialing modems, etc.
But I wanted to be able to call a human and use synthetic voice to deliver a message (and accept touch tones back and turn those into typed commands or something).
To set the context, this was a big deal because most people still had rotary phones! They sold these little modules you could hold up to such a phone to play touch tones on such a phone. Pretty unreliable, as I recall.
Anyway, I had already been through my first couple of passes implementing the 'Naval Algorithm' (turns written english into a series of phonemes) and was a big fan of this 'adlib' chip that basically implemented that 'excitation oscillator into a resonant cavity' math, but rather than you providing all the tweaky little numbers, you just say "play phoneme 3" and it would emit that sound.
Anyway, I loved that chip, and I am pretty sure I snuck it onto the board anyway, but it wasn't considered intelligible enough for our needs.
So we hooked up with Texas Instruments, who were in the process of trying to commercialize their SpeakNSpell technology (the problem being that memory chips were too small to just use recorded samples, so they had to take human voice recordings, process them with supercomputers down into filter coefficients, and then THAT data was small enough to fit a reasonable number of words in a reasonable number of bytes.)
We were one of their few customers for the feature (and they charged about $500 a word (about $1000 a second, I guess) to digitize the words you asked for. You got a discount if someone else had previously asked for the same word. They had some radio announcer do all the recordings, so you could sort of mix and match sets (of course, that always sounds a bit horrible)
We then solicited word suggestions from OUR customers (banks, oil wells, automated machinery wherever it might be.. traffic light controls) and ending up buying a hundred words or so, which I then hand-burned into EPROM and coded up a player for. Now, hmm, why do I not remember the TI chip I would have needed to use for this? the math was NOT done in my 6502... wow, there must have been a chip, but I don't remember it at all! Just that adlib chip, my true love.
But their chip was the same deal, excitation oscillator driving a simulation of a resonant cavity. Only theirs was based on 'linear predictive coding' (LPC) and the description of that voodoo always seemed beyond me. Much like designing multipole filters. Probably EXACTLY like designing multipole filters. I remember the architecture was basically a ladder of paired nodes.. maybe six rungs in the ladder? I imagine each pair, was a pole and this was a six pole filter where each pole could have a freq and a behavior, and that if you picked the right constants, you could imitate a moment of speech.
But coming up with those constants was apparently very difficult and expensive. Gotta pay the DJ for one thing.
Oh, and they could only do that one guy's voice (they said) since the engineers had all been men, they had built in a bias accidentally such that it couldn't do lady voices (they said). I sort of believe that, certainly that the magic mojo for encoding a different voice might be expensive to work out.
One of our customers was a bank in Saudi Arabia, and they asked for a lot of custom words/phrases in arabic.
Of course neither we nor TI spoke any arabic (then being then), so there were destined to be quality issues. We found some local college professor who poke arabic and recorded his voice and sent it to TI for encoding. We got it back, I burned it into EPROM and verified it 'sounded like arabic' (or at least plausibly like the recording)
We packed it off to the customer and waited for their excited pleasure.
But instead we heard: "This is unacceptable. Clearly these words were spoken by an EGYPTIAN! Our customers will not accept this!"
So, imagine my surprise in learning a new way for people to be prejudiced against each other, but don't let me cast stones.
They were actually pretty upset, but we arranged for them to send a speaker of their own choice to us to be recorded and we would do the recordings over again (and any EGYPTIAN bank that needed the words would have gotten a discount! :-)
So this guy came out and aside from the recordings, I was also his 'entertainment' and it quickly became clear that I was a 'lot' less entertaining than he was used to.
But in small world, it turns out he went to college in the smallish city I grew up in, and we had a great chat about that.. Then I think he went out looking for women or something. I do not know.
Anyway, they were happy with the new recordings and bought several of the units, so mission accomplished.
----
But, really, the speak and spell technology was outstandingly weak, and it's amazing we couldn't just get together a few thousand bytes for nice recordings.
But welcome to 2018, where I am ALMOST on the verge of making a weak imitation of speak and spell, using what is effectively a supercomputer compared to the resources they had.
And I still don't really grok the voodoo of filter design. But I accept that you can do amazing things with a weighted chain of memory cells.
Inside my biquad filter are only 4 memory values, yet that is enough to remember all of the history that led to the filter getting in its current state. Not all that different from neural net weighting really. Just add feedback.
[end reminiscence]
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
Watch me notice and fix a bug, in excruciatingly real time!
OK, that link was killed by YouTube. It's actually a compliment! They felt my 'groove' playback of a vocoding was close enough to the original to be complained about! Considering that's a recreation of sound 'from scratch, using no actual parts of the original', that's pretty amazing. They also declared my synthetic instrument (patch) to be 'an oboe'
So here is the link to a revised copy of the film where I mute the offending bits. At least I am getting experience with Premiere Elements. I can cut and paste clips, adjust volume, and add text and narration overlays. I'm still too lazy to do it WELL, but it's less of a panic to open the program. (Adobe just doesn't think like I do, when it comes to UIs. I'm sure I'm the wrong one, but we just disagree on things).
Now I need to make the official 1.07 release video.
Sigh, this video is taking forever to upload (it's an hour long! now with several minutes of silence!) so I don't know the url and will likely post this without it by mistake....
posted
I feel inspired to blabber, but have nothing really to report. Work-work has been busy-busy and my body is needy.
I taped some more sessions of just playing with the game, then watched them and noted all the bugs. And tonight I formally fixed a few of them, but they aren't fun ones. They are minor visual blemishes that end up requiring a lot of research and resolving tricky race conditions, and lots of builds required to tweak them into being useful.
But I finally got to the bottom (I say) of stuck notes. I still have some issues (like if you turn on mute in the middle of a note), but mostly it is 'correct' now, and I improved the TRACK EDITOR so show duration c hanges in realtime, and impose limits to prevent making 0 length notes. And I reasoned how come a note following another would sometimes go invisible. Oh, and after coming up with a complex solution, I found a simple solution instead, so that made me happy. Not perfect, but you should avoid overlapping notes in the same track.
I improved the arpeggiator speed, and the vocoder is now fast enough (with retriggered note detector.. did I talk about that?) to capture Despacito (on the recorder) with few missing notes.
as to retriggering... I have these zillions of filters, one per note pitch that I can detect, like a harmonica.
If you look at a single filter's output and play a song through it, only the notes matching that filter come out the other side. So it's like you are listening to a one-key piano and someone going dink dink dink. There is only one dink at a time, but the dinks can be as long as you like, with as much pause between them as you like. When I see a rising edge in 'the dink stream' I think 'hey, that might be a new note starting' and then I switch to a state that debounces that. If the note energy remains for long enough, I announce a note start (and use that original timestamp). Then I go into a state that waits for silence, and that also goes in two steps before you are back to state 0 waiting for new energy.
So, the point being, I could not detect a second dink reliably until AFTER the first dink had gone completely silent. But in fast music, it's more likely that the second dink starts before the first dink hits silence.
So I just added 'rising edge detection' in my 'waiting for silence' states. and that worked like a charm. In theory, it also means the notes coming in from the vocoder have 'velocity' info available. (unlike using the tablet as a keyboard)
I probably already talked about that, sorry
I still need a color palette for my chording keyboard, and some simple symbol that means 'chord' (and is ideally just a rectangle), and it might still need some work.
The arpegiator starts the moment you tap the root key, and then arpegiates at exact beat moments from that start
However, that means if you tap the key a bit late, the whole arpeggiation is a bit late. and then if you change to a new root, there is a gap/syncopation in the arpeggiation. And I don't know how much of that to put down to 'I am a bad player' and how much should be 'make them sound like they know what they're doing'
I almost feel like I should 'weave' the arpegiations when moving from one root to another (like auto invert the first pass through the chord if the step is too far from previous arp note...
oh wow, now I have to try that.
* pick next note of arp using existing rules * compute num halfsteps it is from prev note * if > 12, subtract 12 * if < -12, add 12
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
This is actually kinda old already, just some more messing about with the vocoder, followed by a surprise reveal of the day zero Chording Arpeggiator (it's already long past this stage.. I have a bunch more footage of the evolution).
It can now arpeggiate quite rapidly with solid equal-length notes (unlike in this video where it was still very loosey goosey)
And I added an option to let you feed the output of the synth/sequencer into the vocoder (and then back into the sequencer), so it can play a song and record a new track into it from the vocoder scanning what it heard. (shown in a later video, but trust me, it's cool!)
Anyway, physician heal thyself: this gives the game a way to do some more targeted guess-less testing of itself.
I have another video coming where I play a test song using different instruments (with different harmonic content) into the vocoder and record the result, comparing to the original (still on track one) and you can clearly see why it's so important that I accurately strip out 'the harmonics' and try to ferret out 'just the fundamentals' during the vocoding process.
To that end, I've tweaked my harmonic suppressor a few more times, hopefully always in a forward direction. It's possible I should just be brutal and if I don't literally hear something like a flute, I just play note nazi and declare "nothing above octave 5!"
But first I have to have a stab at a 'cymbal crash detector' since that's the source of many issues.
ooo, as I type that... omigosh, I think this would work. Ignoring background noise for a moment, all the vocoder ever sees musically is notes and their harmonics, and those all appear at rigid offsets from their fundamentals.
My harmonic suppressor already goes through and marks all peaks it feels are just harmonics.
What I have been ignoring is 'the space between the harmonics' That is to say, if I am playing a single note, so there is but one fundamental in play, I have this rigid array of harmonics at predictable half-step offsets. If I see energy (also) at any of those, I can tell myself "oh, that's probably a harmonic of that fundamental" and ignore/process it.
But if the answer is: "no, this is not the offset of any known harmonics to any audible fundamentals, and yet I see a lot of energy here anyway..."
Then I can increment a counter "I vote for calling this a-tonal noise", and if the counter reaches N (over the course of scanning a single spectrum), than I declare "by george, there is a cymbal crash happening during this spectrum"
Coool, so basically, since I already have all the numbers, just have to add up the energy from all filters not already assigned to fundamentals or harmonics there-of. If the aggegrate energy > X, then its some sort of percussive event, and maybe it's worth distinguishing, I dunno, a closed hi hat from an open one... try to work out the four most popular midi percussive effects and then hardwire the detector to claim it heard one of them.
But, and here's the point, it would prevent it from turning into a regular note on some nearby innocent instrument.
I like simple. Hope that works. I want to finish that 'several instruments recorded to different patches in a single vocoding' work.
Well, YouTube has provided a url (but upload still in progress), which I hope is not blocked by the forum not accepting https...
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
I was bemoaning the loss of "Text Edit for Google Drive" and how much of a gumption trap that created for my starmap development (a lot).
Eventually a perfect text editor will appear on Google Drive again, but until then I wanted something that worked. And ideally worked on Kindle (without needing Google Play Services).
I looked into DropBox, but frankly it felt very heavy. Large demand both on the end user and me. And more powerful than I need.
So, I originally poo-pooed the thought of importing from the local device filesystem (too techy), thinking the google drive approach was more cool and 21st century ish and artist -friendly.
But hey, do what works. So I added a new command to the STARMAP OPTIONS menu
"Import Starmap from Local Drive"
So this allows Kindle users to be Starmap developers (and also works on Google devices with GPS).
STARMAP DEVELOPER WORK CYCLE:
* Use your favorite text editor to make your starmap file on your computer (and I imagine this is also mac-friendly).
* Save your work on your PC somewhere you won't lose it, and can maintain your versions as you see fit.
* connect your device (with synSpace: Drone Runners installed) to your PC with USB or whatever other file sharing technology you have (bluetooth maybe).
But you have to be in file sharing mode, like as if you wanted to pull videos and photos from the device
* Locate the proper directory (see below)
* drag your file from the PC to this directory (this is basically one-way, so don't lose your original on your PC)
* Start up Drone Runners on the device and go to the options menu/starmap options, and tap "import starmap from local filesystem"
Currently this will look for a subfolder called "starmaps" and the file "starmap test.ini" inside of that.
I'm feeling kinda lazy and don't want to add a chooser that would let you pick from available files, so I might just demand you use a special file name and can only have one importable loaded on your device at once (which, honestly, would save you a lot of mental hassle and help you avoid forgetting where your master copies lived...)
Anyway, it will work at least that well in v1.07
--- So what is the appropriate directory?
Well, I think this varies from device to device, but when I connect my Kindle to my PC with a USB cable, and set the kindle into a file sharing mode, I can explore its directories, and the top one is called:
internal storage
I open that one, and there are folders with names like "Photos" and "Music", so I (using my computer) just created a new folder called "Starmaps" and then dragged my test starmap into that folder.
And that worked.
It's a bit confusing since technically I use what is called 'public external storage' (which allows your computer to read and write the files) but the volume is called 'internal storage'.
On another Kindle, where I have both the built in memory, and an add-on memory chip, it shows both an internal and external folder, and in that case (I assume) I would use the external folder.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
I also got a new toy ("Maker Hart Loop Mixer") which is an audio mixer panel that is cute as a button. It can mix 5 stereo inputs into three stereo outputs with individual level controls on input and output.
The inputs have a 'gain' knob which allows them to match the impedance of either a MIC or LINE signal. I think the outputs have to be all considered LINE out. (well, other than the headphone jack)
ANYWAY, it's cool. You get five sliders to mix the inputs (each with its own balance control if you want to nudge them left/right a bit -- which assumes left and right are actually the same on many channels)
The headphone out gets a private volume control. The main out gets a master level control, and the record out comes out pure mix with no attenuation (assuming the recorder has its own front end gain adjustments)
With this puppy, I am able to mix the computer and tablet signals back and forth and listen to both, in stereo, simultaneously, with a single set of earbuds. My 'two pairs, one in each ear' technology is great and all, but requires a lot of 'ear hopping' and earbud cable management that also was a gumption trap for me.
Anyway, cool little unit. I actually shopped a long time for this one. I'm very happy with the mix of features I got (they have other models, optimised for other situations).
I should point out that this mixes TRS (tip ring sleeve) 3.5mm connectors. I actually had to use a splitter on the TRRS (mic + left + right) so I could route L+R into the mixer, but send the mic directly to the earbud (with another splitter to glue the signals back together). And that worked great.
I got some new TRS cables of perfect lengths made out of some super supple silicone like material (and the splitters use this sort of 'flat wire' which is also very satisfying.)
My only remaining issue is that connecting the tablet's USB cable (connects to PC) injects a bunch of noise, so I pretty much have to unplug it most of the time, which means the tablet goes long periods without being charged, and synSpace is a real battery drainer, so it limits development session duration.
It's POSSIBLE that the noise requires the presence of my magic stepdown cable (converts line out from PC to mic in of tablet) and maybe the new mixer set up would let me do that without the magic cable, by turning the master mix level real low... and maybe, in addition to working, it would be less noisy.
But I doubt it, I think I need a really great copper ground rod and then to make sure my PC and the tablet are solidly grounded and all cables are super shielded. And that's not going to happen, so I will continue to pull the plug when the sound is bugging me.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
Not sure if there is much novel in it. I am still catching up on dev sessions, which I pretty much always record now, since it really helps me catch bugs and design flaws. This is pretty highly edited down from about four? hours of dev time.
* example vocoding session of copyright-free music "Jazz Tape" * example sequencer recording of vocoder output * example playback of spectral, raw sample, and sequenced * replay of sequenced on different Patch * short lecture on what a note is in my system. * contour generator noodling * attempt to create a voice-like Patch using glissando * sequenced groove playback of Sally's Song on glissando voice patch (like a friendly swarm of bees). My glissando only works musically when you keep to one note at a time.
----- this weekend, in addition to having an intense cold, which I have actively treated with Emergen-C, and actually feel better with fewer symptoms. So I will keep that up for however long these beasties usually live.
Anyway, now that I have that loopback that allows me to play the groove directly into the vocoder and back out into the groove, I can do all sorts of perverse experiments (what happens if you encode the same song over and over... does it approach some equilibrium or chaos?)
Right now, I am doing it for diagnostic purposes, but perhaps there is a positive musical outcome. It definitely generates an interesting 'chorus' effect. Or... *can* generate, I mean. It can also generate chaos.
But I did some experiments:
* looked at the classic waveforms to see they have exactly the harmonics the books tell us they do.
* did some tests of the Filter (I mean the synth Filter Stage) since I remain unsure if I like my implementation of 'resonance'
I think I have that on a 'linear' knob though, and perhaps I would be happier with a logarithmic one.
My Center/Cutoff freq control seems to work in an intuitively natural way, but I am constantly unsure of exact what's going on with my 'Q'.
I changed the Music Oscilloscope button (to the right of the Patch/Sqeuencer selectors) to be three state. OFF/ FIXED / AUTO
AUTO is the new mode and it just scales the signal so it doesn't shrink to nothing when it is just quiet. This allows me to see the Q value I am sending to the filter stage, but not clearly enough yet to understand if it is wrong or not.
I think I am using the Q offset knob as a sort of center Q value, and then the LFO oscillates around that value, and the dB attenuator controls how far the oscillation goes. (and I bet that is also linear and should be log)
But when the envelope drives Q directly (no LFO), I think the envelope just goes from 0 to some fixed max Q value (like 100)
So, aside from lin/log issues, I think I can generate the sort of Q changes I want to see. But if I lower the center-Q, the bottom part of the LFO wave gets ever closer to 0, and once Q goes negative, all bets are off.
Hmm, I wonder if I should be using the log() of my energy... my spectrum display is, come to think of it, rendered linearly. And my peak detector works from that linear data. It would be more 'natural' (but possibly not better)
Ditto for the contour generators which also ought to be log if they are not. but I always say they are, so lets hope I am not lying. The part you edit can be linear, I just need to convert it to log equivalent before letting a note use it to modulate its loudness.
And I guess that could be an option.... that could make it back compatible..
thought police check:
vocoder is sending two notes one is at 20M energy the other is at 1.8M energy
log(20M) ~ 7.x log(2M) ~ 6.x
And that's the relationship I would like to see. I basically do that today, but by comparing the ratio of values and looking for interesting ranges. But I think the code would read more clearly if these were log.
for the actual spectrum display though, I can see wanting it both ways. What I do right now has nice sharp peaks, with the defect that notes you perceive to be 'about as loud' can show up as tiny dots in comparison (10x instead of 1.x x)
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
Small recap, you can make your own maps and share them with others. Each map is defined by a single text file.
The author of this file, copies it to their tablet/phone via USB or Google Drive. (Bluetooth too, probably. I dunno. That's *your* problem :-)
In the OPTIONS/NERD menu are two commands:
IMPORT STARMAP FROM GOOGLE DRIVE IMPORT STARMAP FROM LOCAL DRIVE
Hopefully I can come up with a better name than LOCAL DRIVE. Anyway, if you are using a non-Google phone, one without the Google Play Store (For example, a Kindle Fire HD), then you have to use the LOCAL DRIVE option and will have to work out how to use USB/BLUETOOTH to get files into your device.
(I use a USB cable and it's identical to dragging videos off of or onto the device. I have my device in Developer Mode, but I don't think that matters for this, you just have to set the USB mode to "files" as opposed to "just charging")
CODE EDITOR FOR LUA
So, If you use Google Drive, there USED to be a great generic text editor called Notepad DRIVE. You can use ANY text editor, but it has to be able to save an actual text file and not a 'google doc'
If you're on Kindle, then I suggest using a comfortable editor on your PC, then copying via USB to the device. I have just started using ZEROBRANE STUDIO which is basically a Lua IDE, but not overly complicated (I guess you can trick it out later)
ZEROBRANE can also execute lua code. It's super simple. It's your basic code-completion editor with tabs so you can flip between files. Press F5 or F6 while one of those files is selected, and it gets 'executed' (it assumes the file is an executable lua program, and any comments are inside of block comments --- just like a starmap!)
Of course, a synSpace starMap can't run outside of the game. Well, it can run fine, actually, it just can't control any game elements or get any data from the game. So maybe you can use some sort of unit testing code in zerobrane. Actually, just running it without an error popup would be a good sanity check before bothering to drag it to the device.
But once you're ready to play your map on a real device, you
1.) save the copy in your special master copies vault that you never accidentally delete or forget where the latest versions are...
2.) drag it to the appropriate folder on your device, using USB, BlueTooth, or any means available. It's technically stored in 'public external storage' though that means different things on different devices
3.) Run the game and select OPTIONS/NERD/IMPORT FROM LOCAL DRIVE
4.) that will load your map and start a public server for it (yep! no solo testing. how rad is THAT?). You will be the only one on the server, and your map will have a temporary file name.
5.) Anyone who enters, will automatically receive a copy of the temporary starmap (but won't be able to save/clone it until it has been formally released)
You can then repeat that indefinitely. Once you have a version that you really like, and want to formally share with others, then you import it one last time and then CLONE it, giving it your very best name (actual names are inside the file, but people will also see this little name in their chooser list, so be nice and helpful)
Now that you've cloned it, it has an official asset id that is unique to that version of that starmap. This is in your local database until you delete it. You then select it like any other starmap (no more importing required, unless you're working on a newer version, or another starmap)
So anytime you play your map on a public server (which is 'always'), anyone who enters will get a nice copy of it. And anyone who starts a server with that asset tag, and anyone who enters that server, the system will guarantee that all players have the identical version of that asset. (peer to peer, as necessary)
Which means even if you delete a 'popular' starmap, it's likely to come back unless you scrupulously avoid selecting it in the map chooser when you see other people playing it.
I prefer to think of that as a form of holographic memory that helps preserve your art.
----------- Executive Recap
* Lua is fun, ZeroBrane looks fun and 'easy' * Kindle Fire HD Players can now make starMaps (as f version 1.07)
If I have a secret mission to encourage people to get into coding/scripting, I'd say ZeroBrane drinks the same coolaid. Coming into both at the same time (ZB and Drone Runners) is probably confusing since your code will be able to be run on two different computers and you have to understand what that means exactly.
If ZB turns out to be amazing, we can probably work up some sort of test harness that let's you test stuff (fake game events and such). AMAZING possibilities...
But right now, the value to StarMap developers is just the ability to edit a text file and then drag it to your phone/tablet. Whatever you like, that can do that, is fine.
Hey, there is a video possibility there :-) One that might talk very little about music. Then again, a Music-based StarMap is on my list of deliverables, so maybe there is no escape. Plus I need to add 'instrumentation' for all the new 1.07 features, so your lua program can do cool stuff with the vocoder/sequencer/synth.
Let's see if my wife is peeking at what I am typing. She doesn't know that I ordered another musical keyboard toy thingamy (a 25 key usb midi controller with about 2 octaves of velocity sensitive keys and a bunch of LED-lit knobs and drum pads and such, in a cute little form factor, and completely powered (it claims) via USB.
Which means now, I *have* to implement MIDI (incoming at least) in the game. But I will continue to refuse to spit MIDI back out. I don't know why, but my thing isn't MIDI. It's... something else. In my heart.
It's a space game. It's just a 'toy' DAW anyway. Just a pile of knobs to twiddle that may or may not generate the next cool sound. Though I am finding it to be a pretty useful "Music Oscilloscope" so is that an existing product category?
And while my laser focus is drifting, I've been looking at USB Oscilloscopes (I have to keep telling myself I have zero need, but as they get cheaper, there will inevitably come a time when I will not be able to resist temptation and my closet will get another unopened box). But they are getting cooler.
Actually, I lie. The USB oscilloscopes SHOULD be fantasting (a metal box with solid preamps and a USB3 to a tablet/PC with all the display/smarts in it.) Metal Box has to have a large sample store and high bandwith xfer, but tablet/PC can do the really long term storage
I think I would like a knob or two on the metal box (I mean, I don't want to use touch screen for some of the controls). But it occured to me you could just design it so that it could use a MIDI control if you wanted physical knobs, and I bet there is already a category of MIDI control that is just a small handful of knobs. Anyway, using midi controllers as add-on options to things that are not music-related.
ANYWAY, the USB scopes I have seen so far are generally * slow (My snobbery dictates a solid 50MHz 1GBS/s min) * expensive (My seduction price is like $50) * laggy (from touch to update almost a second)
Now I don't want to miss the opportunity for new pardigms (like pinch-zoom) and the thought of just recreating a fixed scan oscilloscope when there are no limits to how the signal *could* be displayed, well. It just seems like doing that 'in the tablet with an openAPI for the metal USB Scope 'brick')' would encourage cool things to happen. There is a fairly nice (but laggy and expensive) kickstarter usb scope that I guess happened a couple years ago, and now their sales page looks like it doesn't get visited much and maybe your $400 isn't safe on that page (it probably is, but way outside my seduction range)
Bottom line, I don't need one at the moment. But I *like* them. Used to use them *all the time* and these are the nostalgias that tear at me :-)
ANYWAY,
KINDLE FIRE HD CAN NOW BE USED TO DEVELOP STARMAPS FOR SYNSPACE: DRONE RUNNERS.
there, laser focus at the finish!
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
This is a research session (edited down) exploring the effect of a lower 'Q' value in the vocoder filters and some use of the vocoder as a MUSIC OSCILLOSCOPE to analyse harmonic content of individual notes.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
My toy MIDI controllers came today (wewt!) One had a bad USB cable (apparently pretty common with this product), but both work once you get past that.
The one with the bad cable also has super hard to press keys. Not sure if that's just me or not.
The other one feels more typical, has working velocity sensitivity and a bunch of lovely control buttons I am hoping to send to synSpace someday
So I looked into "what API support is there for MIDI these days. Used to be pretty sparse and third party and, as I mentioned, I avoided it.
But as of API 23, there is a lovely media.midi class and I was able to do 90% of what I needed in a hour or so.
Unfortunately, Kindle Fire HD tablets are only running API 22, so the class is not available where I want it to be.
So that kinda sucks, Kinda lessens my enthusiasm, but I guess it might work on my phone, if I get an adapter cable (darn USB-C)
I just played with my new keyboards on my PC with 'dexed' and that was lots of fun!
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
So I guess 'midi controller support' will have to remain a pipe dream for a bit longer (until Kindle gets that API.. I don't see much need for a GPS-only solution while I am the world's only user :-)
But I have a new 'theory' related to singing, another subject over which I know nothing, and yet about which I seem to have a lot to say.
I have been analysing a fair amount of human singing, mainly with the goal of working out which note pitch is being sung.
The reason this is challenging at all is because the voice actually is full of harmonics, some of which are much louder than the fundamental.
So my harmonic suppressor has gotten fairly effective at finding the fundamental and then filtering out the harmonics, so only one note is 'heard' instead of it reporting 'a chord of harmonics'
Hence, when you sing, I more or less accurately report the lowest pitch in your voice, the fundamental, the pitch that is actually coming from your vocal chords buzzing.
But as I do that, I am constantly seeing situations where the second or third harmonic is so much louder than the fundamental, that the audience *has* to be reacting to the harmonic and ignoring the fundamental.
And hence, in that case, I am incorrect, and should be encoding the harmonic instead of the fundamental.
Enter my theory of singing.
If you start by talking clearly, and then raising the pitch of your voice until you can no longer talk clearly without 'going into a falsetto', then you have reached the upper comfort limit of your vocal chords.
To 'sing' a note any higher than that REQUIRES THAT YOU EMPHASIZE A HARMONIC. i.e. you hold a lower note, but squeeze your 'throat shape' until it resonates at the second or third harmonic of that note, which can then be much louder than the fundamental (though some of that also gets out and shows up on my scope)
So a good singer can sing the same note in several ways. Just plain 'pluck' their vocal chords at the desired freq and open throat completely, or pluck at half or third of the desired musical note freq and increase aspiration to play a 'harmonic' on their vocal chord (much as you can pluck a harmonic on a guitar string without changing the length of the string by fretting)
This also gives you another 'fingerprint' technique to identify individual voices by which notes they 'have' to use harmonics to hit.
Now I have to check out that F# on the recorder again and make sure I know which octave the perceived note was in... I assumed the fundamental, but now I think I was probably wrong and that was an F# that required an octave harmonic from 'special breathing' by the performer.
----
I'm editing down another old development session today. A particularly boring one with a ton of repetition, back when I hadn't grokked the fact that the third harmonic of a note is also a perfect fifth above its octave harmonic. That's cool.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
Still catching up on progress videos. This one talks about 'alien voice' design (in game chat) and a little demo of the new chording arpeggiator (that will be featured more in the next few films as I catch up with development from a couple months ago :-)
In fact, this was from a medium length development session where I didn't really achieve much, and then had to edit 3 hours of video down to 30 minutes, and still didn't end up with anything all that compelling the look at (and the editing easily took twice as long as the original experience, so not a good use of time).
Still, all that reviewing allowed me to address every little bug that came up, so I guess it wasn't a complete waste.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
Ooo, I added something last night that I suspect is what they call a 'compressor'
My final sound output has to fit inside a 16 bit number, so is limited to the integer range of +/1 32767 ish.
My math is all double precision and goes wherever it goes. All my stages try to be amplitude neutral so the final signal from the synth naturally fits in the available space.
HOWEVER, inevitably, there are things that make that even louder (several notes at once, simultaneously in the loudest parts of their envelopes) or, as was the case last night, the reverb unit (where I was fixing that 'first echo too loud' bug)
So everyone once in awhile, my numbers would be outside the legal range, and I would react in two ways: I would ultimately clip the value to something that fits, but creating an audible click in the process, plus I would signal my automatic gain control (AGC) to scale things down a notch so it doesn't happen again (and then slowly crank the gain back up during quiet periods)
The AGC is pretty nice, but it can't stop that first click reliably, and so I actually had more clicking going on that I'd realized.
ANYWAY, so I had this idea, which in retrospect I am pretty sure is just 'compression' in the sound engineering sense.
I picked a level (30,000 about 10% down from max allowed value). Any synth output below that gets passed unchanged. But I tell myself I have a worst case scenario of 4x max loudness (which is just an estimate, so this could still click if I exceed that). And then anything above the 90% mark gets scaled to linearly span that reserved top 10%
So loud signals are distorted, but they were clipping before, and this sounds better :-) In fact, the reverb sounds amazing. A little too clear now, in fact, and I can hear the 8 individual feedback taps now, and it kinda undermines the effect. Need more taps. Though actually I had completely forgotten about my reverb plans (dynamic tap points, flange, chorus, etc)
But here's the kicker. I demonstrated recently that if I use the new arpegiator to play my fm_tubular bell patch (with long notes), it took almost no time (8 notes?) before the sound broke up in a way I interpreted as 'cpu can't keep up with all the math'
Well, after switching to a compressor, I can do 4x arp at 200 bpm with 3second notes with no screwups at all. I have no explanation for how clipping samples got turned into the defect I heard before. unless it was just triggering the AGC and I didn't notice.
Anyway, I hope that remains true as it's cool that it can play notes super fast now. Still can't vocode that fast though.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
This one mostly shows flaws in the UI of the new chording arpeggiator and what I might do to improve it.
In general, I want to replace 'tapping 15 times to get to the value I want' with 'tap once to see a menu of seven choices'
Then I can use the I through VII chord keys as the value buttons and the seven regions on top to control seven properties.
speed, latch, pattern, key, mode,. home...
home... I had this idea,, maybe it's stupid. You are in the key of C playing along, and you are playing .. say.. an F chord. You would like to switch to the key of X for a minute. Maybe a refrain or something that needs to sound wistful.. And that means you will need access to a different set of I-VII chords. You need to change to a new key, but which one? Once you pick it, you will see the new I-VII chords, but before you pick it, how can you know..
So, here's my thought. I am playing an F which is the IV chord of the key of C, but I want to transition to a key that still includes the F chord as one of its I-VII chords. I have about six keys to pick from. seven probably.
so, being sophisticated in the ways of wistful, I say "I would like to transition from my F chord being IV in key of C to being II in key of x.
So, mister computer, you pick x for me given my selection of the roman numeral I wish I were.
So once you make this selection, you are still playing an F chord, but it is now on a lit II key instead of a lit IV key, and the other keys are all mapped to the new key signature, and you can start making a wistful chord progression in that new key.
And you know you could get back to home key by getting back to F then re-homing it back on IV.
Probably stupid, but I'm always looking for gimmicks to trick me into accidentally making something musical.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
Edited down from about 4 hours of material, it's still over an hour long, with my riveting speech patterns thoughout.
It's mainly doing experiments with the then-new loopback feature that allows me to use the vocoder and synth/sequencer to test each other. Mainly towards the goal of harmonic suppression (as in recording 'just the fundamental notes' of a song, and not all the harmonics)
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
posted
This weekend I mainly looked into why my SAMPLE mode playback was so noisy. Turned out to have several simple causes and now is pretty glassy smooth. I.e. sounds like what you'd expect, instead of completely horrible.
As part of that I added the ability to single-step in SAMPLE mode, which requires looping the samples from one 25th of a second (640 samples) over and over, without introducing a 25Hz click (or worse).
I ended up doing this: * I play the 640 samples 'forward' * when I get to the last one, I start playing them 'backwards' in the other direction * as I do that, I look at the starting samples, the same distance in from the front as I am in from the back * as soon as I find samples whose 'slopes and positions match' (or come close), I stitch them together and complete the move back to the front.
After I did that, it occurred to me I could probably just play the whole range forwards and backwards over and over. Not sure humans would hear much difference if it's a short enough interval.
I think life would be bbetter if I could do 50 spectrums per second, but that still feels beyond the level of device I would like to run on.
-------------------- He knows when you are sleeping. Posts: 10640 | From: California | Registered: Dec 1998
| IP: Logged
Topic Closed
Printer-friendly view of this topic
