/mlp/ — Pony Preservation Project (Thread 160)

FAQs:
If your question isn’t listed here, take a look in the quick start guide and main doc to see if it’s already answered there. Use the tabs on the left for easy navigation.
Quick: docs.google.com/document/d/1PDkSrKKiHzzpUTKzBldZeKngvjeBUjyTtGCOv2GWwa0/edit
Main: docs.google.com/document/d/1y1pfS0LCrwbbvxdn3ZksH25BKaf0LaO13uYppxIQnac/edit

>Where can I find the AI text-to-speech tools and how do I use them?
A list of TTS tools: docs.google.com/document/d/1y1pfS0LCrwbbvxdn3ZksH25BKaf0LaO13uYppxIQnac/edit#heading=h.yuhl8zjiwmwq
How to get the best out of them: docs.google.com/document/d/1y1pfS0LCrwbbvxdn3ZksH25BKaf0LaO13uYppxIQnac/edit#heading=h.mnnpknmj1hcy

>Where can I find content made with the voice AI?
In the PoneAI drive: drive.google.com/drive/folders/1E21zJQWC5XVQWy2mt42bUiJ_XbqTJXCp
And the PPP Mega Compilation: docs.google.com/spreadsheets/d/1T2TE3OBs681Vphfas7Jgi5rvugdH6wnXVtUVYiZyJF8/edit

>I want to know more about the PPP, but I can’t be arsed to read the doc.
See the live PPP panel shows presented on /mlp/con for a more condensed overview.
2020 pony.tube/w/5fUkuT3245pL8ZoWXUnXJ4
2021 pony.tube/w/a5yfTV4Ynq7tRveZH7AA8f
2022 pony.tube/w/mV3xgbdtrXqjoPAwEXZCw5
2023 pony.tube/w/fVZShksjBbu6uT51DtvWWz

>How can I help with the PPP?
Build datasets, train AIs, and use the AI to make more pony content. Take a look at the quick start guide for current active tasks, or start your own in the thread if you have an idea. There’s always more data to collect and more AIs to train.

>Did you know that such and such voiced this other thing that could be used for voice data?
It is best to keep to official audio only unless there is very little of it available. If you know of a good source of audio for characters with few (or just fewer) lines, please post it in the thread. 5.1 is generally required unless you have a source already clean of background noise. Preferably post a sample or link. The easier you make it, the more likely it will be done.

>What about fan-imitations of official voices?
No.

>Will you guys be doing a [insert language here] version of the AI?
Probably not, but you're welcome to. You can however get most of the way there by using phonetic transcriptions of other languages as input for the AI.

>What about [insert OC here]'s voice?
It is often quite difficult to find good quality audio data for OCs. If you happen to know any, post them in the thread and we’ll take a look.

>I have an idea!
Great. Post it in the thread and we'll discuss it.

>Do you have a Code of Conduct?
Of course: 15.ai/code

>Is this project open source? Who is in charge of this?
pony.tube/w/mqJyvdgrpbWgZduz2cs1Cm

PPP Redubs:
pony.tube/w/p/aR2dpAFn5KhnqPYiRxFQ97

Stream Premieres:
pony.tube/w/6cKnjJEZSCi3gsvrbATXnC
pony.tube/w/oNeBFMPiQKh93ePqTz1ns8

Quick thread to see the state of things.

15.ai is now dead forever.
https://x.com/fifteenai/status/2060098921582772567

Is there a dataset describing the scenes of the show, kinda like a transcript but with detailed information on who does what?

>>43289213
I don't think there would be such thing for mlp (otherwise it would be scrapped for data years ago), I guess the closes would be to try rip off netflix and other tv shows that have the Audio Description/Video Description tracks for the blind people, and see if that could be somehow used as dataset in whatever project you are planing to do.

>>43289085
This MF just abandons every project of his, doesn't he
He'll abandon that marketplace soon enough and then announce a brand new project that he totally won't abandon, bros!

File: coolshit.jpg (1.35 MB, 7096x2020)

>>43289238
I doubt it would be detailed enough. I heard multimodal llms can take in video, even small ones like the ones I can run on my machine, so one could theoretically tag the entire show like this.

That aside, picrel is a compilation of experiments I did a while ago while playing with show frame compression. I trained a hierarchical VQ autoencoder that encodes 256x384 frames into two maps: 16x24 and 8x12 codebook indices, each codebook has 1024 entries, and then tried to generate larger map given smaller map using discrete diffusion. It's quite undertrained but I got bored of it. Just thought you guys would appreciate the abominations, some of them are even cute.

>>43289447
I think I saw some models months ago that were able to watch 10s video and describe what was happening in a scene along with any interaction people had with the setting, I wish I could remember that the name of it was since that sounds like something you could potentially reuse for your stuff.
Now that I type all of this, I kind of wish there was a program that could easy way to create automated audiobook from a fic, but that would require for tts model to be combined with some llm to understand which characters are included in the story and automatically swap the voices of character/narrator as well as add any relevant background music and sound effects.

>>43289481
>easy way to create automated audiobook from a fic
I made such app a while ago https://files.catbox.moe/cwj64u.mp4
https://drive.google.com/drive/folders/14zMbURz1SuYNMoewX88EjR8sHEkcaKXa

>>43289508
>if yours only supports cuda 11.x but you still want to run on gpu, run the following inside PVT folder: runtime/python.exe -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
Could it be possible to make it work with the cu116? My system+gpu can't get python to work with anything above that, and Im not going to be upgrading my pc for at least next two/three years.

>>43289524
Replace cu118 with cu116 and try it, should work.

Oh hey, the lights are back on here?

does anyone know if there was an upgrade in openly available musical ai models or is the yue music model from last/two years ago still the "best" option?

What would people think about trying to make some kind of collaborative "album", with Anons posting their pony songs made with whatever ai model/service of their choice every Friday?

>>43291450
The idea sounds interesting. I'm not sure if there are enough Anons who are actively genning to make that system work though.

Up.

File: angrycelly.png (447 KB, 880x706)

Hello! Maybe someone here could help me find a certain audio file I'm looking for. It's Princess Celestia saying "And so, as punishment for your insolence and treachery, I hereby sentence you to death by decapitation! May the blade fall swiftly and your head be raised high as a warning to all those who would dare to defy the will of the crown." I can't remember whether it was AI generated or audio from Nicole Oliver herself or a fan dub, but I remember that the quality was very good. I know I listened to it in November 2024, but it may have been posted way before that. I scoured the PPP threads on desuarchive and Clipper's master files and did not find it.

>>43293091
>The PoneAI drive, an archive for AI pony voice content:
drive.google.com/drive/folders/1E21zJQWC5XVQWy2mt42bUiJ_XbqTJXCp
Try here, if it was genned in a thread then it'll be found in here.

>>43293096
Thank you, Anon. There's lots of good audio in there but, alas, I could not find the file.

>>43293091
I found it! I can't believe I missed it the first time; it's right there on desu in Nov 2024 just like I said, >>41653178
Direct link to the audio, for anyone interested:
https://vocaroo.com/14eyuFuDu0Zs

>>43293091
>>43293256
That sounds pretty OOC for Celly.

nine

>>43293971
plus one

>cant use raw audio clip because singing reverb fucks with the voice conversion
>cant use the DeReverbed audio because random bits of words are just nuked from the clip
dear song makers, plz stop using aggressive autotune and fake ass reverb, its making finding a good song for ai pony covers way more difficult than it needs to be.

>https://u.pone.rs/basikmdv.mp3
>technobridge523 - BLACK SUNSET (Cover)
What a strange feeling, bumping into a brand new ai cover of old original ai song made in ppp thread from two years ago.

What's the latest and greatest in local voice gen?

we are so dead

>>43289079 (OP)
We drove this deep into the heart of the Archives. We thought it was dead. We were wrong.

>>43297041
we are so alive

Gpt sovits on haysay is fucked and keeps giving an error. The only decent one that allows text to speech and has all the good characters.

>>43297664
Hmm, Im guessing HydrusBeta amazon server decided to crash out on that specific tts code , which sucks balls. Im guessing it wouldn't be helpful saying there is local ui installation of the Gpt sovits that can run pretty OK on 8gb vram? worse come to worse, you could always request audio clip form anons in the thread here, as long as it's pony related it would be fine

>>43297664
There was a JSON file that got into an corrupted state. I've regenerated it, and I think that fixed the issue. Let me know if the problem persists.

>>43297705
I've been working around it by using the characters on the tts one and converting them into the other voices with mixed since they require an audio input.

>>43297961
This worked, thank you anon.

>>43297961
Thanks for the fix.

>>43297961
Thanks anon!

>>43297961
noice one, m8!

>>43289085
Didn't he say he was going to open source this stuff eventually? Now would be a good time.

>>43299651
I dont think he ever has, and even posting on ppp was very sporadic in 2019/2020. It would be cool if he had, or at least maybe create the documents showing exact steps of how it was made so people with the know-how would be able to recreate it with the modern tech.

>>43299651
>>43300095
I'm certain he at least stated several times that he was gonna release a paper about his research, which he never did, and likely never will.

>>43300117
Damn

>>43289079 (OP)
hey check out my fuckass tts. I'll release model and stuff for realsies soon
samples:
https://files.catbox.moe/b5asjw.wav
https://files.catbox.moe/2vk68m.wav
demo:
http://198.53.64.194:35029/

File: Mare ooo.gif (428 KB, 235x274)

>>43300755

File: 645067__safe_artist-colon-jackspade2012_maud+pie_earth+pony_pony_absurd+resolution_female_hope+poster_implied+rock_mare_meme_obey_poster_rock_shepard+fairey_so.png (630 KB, 5095x5993)

>>43300755

https://files.catbox.moe/hpg3m1.wav

https://files.catbox.moe/xrrt16.wav

https://files.catbox.moe/uz1fyc.wav

>>43300755
Is this your own arch or did you tune something?

>>43301171
Both. I'll explain later.