As a part of working on the mxTune mod I sometimes get ideas, but have no idea how to implement them. For instance, when playing music in mxTune now, the sound does not fall off at a distance and it's not directional. That is because the MIDI synthesizer is not connected to the 3D sound system used by Minecraft. Minecraft itself relies solely on Ogg Vorbis files for sound effects, records and background music. It handles these very well thanks to the underlying sound libraries and their own sound handler code. The fact that resource pack makers can replace sounds, textures and models is wonderful feature. But modders of course can, with some sweat and effort, and decent modding libraries and resources can go beyond what vanilla offers. However there are usually some limitations you stumble upon that make you want to give up sometimes.

Back to the issue at hand. I wanted to see if it's possible to link a MIDI synthesizer into the 3D sound system. I had experimented with using codecs for the IBXM MOD Player some years back so I knew how to add a codec to the sound system to enable it to play file types other than just Ogg Vorbis. But with the addition of resource packs and a rewrite of the sound system, the .ogg file extension was hard coded in the code. But fortunately there are Forge events for sounds. With those it's possible to intercept a sound and make changes to it. That's a key ability I need because I want to introduce my own file type. This is required so that the underlying sound libraries will associate my file type with a custom codec.

An audio codec allows the underlying sound system to recognize and play different sound file types. For instance OGG, MP3, WAV, FLAC, etc. For my purposes I wanted to use a MIDI type, but I don't want to use the standard MIDI behavior of the sound system. It will play a MIDI file, but not through the 3D sound system. Instead it uses the javax sound libraries. As such it totally bypasses the Minecraft sound settings for volume, and the in-game 3D audio effects. I can override the MIDI extension and use it, but my MIDI file/sequences contain mxTune specific information though technically they are a standard type 2 MIDI file.

Due to the way Java and Forge mods work, it's pretty easy to add some existing libraries to your mod that offers some feature or API that either you don't know how to make yourself, or maybe saves you a lot time. That allows you to focus on other features of your mod. Writing a MIDI Synthesizer is NOT something you do in an afternoon, and my current level of coding and audio knowledge are not up to the task, but I know enough to be dangerous and don't like to run away from difficult problems. So I decided to use a library I stumbled upon while researching HOW I might write a synthesizer, or maybe base one on a MOD player. The library I found is called Liquinth. It's a real time digital analog synthesizer written in Java. It's interesting because it can be used standalone or as a VST instrument using via jVSTwRapper. I'll let you research what those are if you are interested. For my purposes it will serve as a compact simple way to test if it is indeed possible to link it with the 3D sound system.

The first experimental codec just played a pattern file type .PAT. This is simple format the Liquinth author created for testing his code. It's also the format used to save and load the controller settings that affect the timbre of the instrument. The code is quite simple and does not integrate directly with the Minecraft sound system, but instead uses reflection to get a handle on the underlying libraries Minecraft uses. There is an item in mxTune named the Magical Music Item. It uses this first experimental codec. When activated it plays a short chord progression at that location. Walking away you will notice the volume level decreases as you move farther away from the activation point.

This first experiment gave me hope, but at the time I had not yet figured out how to use Forge sound events to change the sound type, nor did I fully understand how Minecraft's sound system worked. So I worked on other features for a couple of months.

The second experimental codec was born when I had a flash of inspiration and eureka moment after reading the MC Vanilla sound code, and playing with the vanilla MovingSound class. This codec plays a MIDI file created with the mxTune MML to MIDI parser code. The codec contains a MIDI sequencer that opens a channel to the Liqunth Synthesizer and plays the sequence in real-time.

The simple demo in the attached video demonstrates the directionality and attenuation of the music at a distance of a moving music source.

This is not yet part of the mxTune codebase. There are some issues that need to be worked out.