The thing about me is that:

• I like anime (of course)
• I used to have the habit of following illustrators and archiving illustrations that I like (!). I don't necessarily do this nowadays but back in undergrad school days I use them as wallpapers.

And because of that I've accumulated about thousands of images. I randomly selected about 1.3k high-res-enough ones and that'll be my training data. Naturally I won't be releasing this dataset because:

• It's been so long, and with people changing names and quitting all the time I can't attribute the artworks to their respective authors any more.
• Even if I can, distributing the arts without requesting for permission is definitely copyright violation.
• And of course there's the problem of it being used by others for GenAI, even if I truly only intend to do (relatively) harmless things like super-resolution with it.

And then I realized: I can just steal from the people who posts AI art on Twitter! I definitely have zero guilt stealing from them, and if anyone came after me for stealing "their art" and violating on "their copyright" I can simply tell them to fuck off, because of what difference would it be to simply prompt a generative model myself and pick an amount of the generated result myself? I can distribute the dataset on HuggingFace without fear as well - what possible measure and moral high-ground could they ever have above me? It's just thieves calling others thieves! Not only am I going to steal, I'm going to steal specifically from those accounts that posts AI art but says 無断転載禁止. Heck, I might use this to train GenAI models too, they can all go fuck themselves… The only thing I will have to be worried about is the fact that AI-generated anime pictures tend to follow only a handful of styles and I might not get enough texture varieties out of this.

And from that logic a dataset is born. If you're familiar with these kind of things (which I unfortunately kind of am), you'll quickly notice which franchises and fetishes (!) which account likes, e.g. this one loves big breasts, that one's a competitive swimsuit fiend; this one's a hardcore Umamusume fan, that one watches Hololive, etc.. The One Big Issue behind absolutely all of them is that regardless which concrete art style the images are in and which topics the images are about, they are all repetitions of the same level of good-but-subpar idea. White long hair, bikini swimsuit; pantyhose, glossy eyes, bedside-look pillow hugging, Shintoism priestess wear; highschool uniform, cheerleader top & skirt, the same head tilt, the same innocuous smile. It's all the same thing, the same anatomy, the same composition, the same implicit scenario, the same posture, the same background, the same rendering, repeated again and again and again and again, and at the end of all this you'd be like, "Man, what kind of quote-unquote 'creativity' is this? These people can't even do 2D anime porn properly".

It really reminds me of that one AI-generated scene I've seen that somehow got popular on Twitter, which was supposed to be an excerpt from a fictional dystopian show (by which I mean the show itself is fictional i.e. non-existent, not the story being a fiction). In said dystopian show, poor people are forced to live within a giant glass tube which has an internal temperature of minus a fucking bazillion degree and was built right in the middle across the normal temperature area where the rich people lives, and also the poor within the tube has to fight against bears (?) and tempests and avalanches that somehow only occurs within the tube and fully-restrained within the tube (!!!) while the rich people would just watch like it's some kind of shopping mall Christmas performance; and let's not forget, because the poor needs to deal with snowy-mountain-like environments, they have snowy-mountain-like equipments, like pickaxes, which they somehow didn't use to break the tube open. It's the stupidest fucking idea I've ever seen for a dystopian story. Are these kind of people supposed to be the new "aspiring creators and artists" generative AI was supposed to enable the creation and cultivation of? What happened to just letting them suffer the horror of not being known for shit (until they have actual good ideas and actual skills to carry out said ideas)? One thing about generative AI that I suspsect is that people almost always seem to think they have the brightest ideas about things while being completely oblivious to how stupid most of the ideas are, and the AI they use was just powerful enough to carry them out in a fashion good enough so that they can comfortably (consciously or unconsciously) skip the step of thinking them through. Does the "poor people climbing through the tube" video clip looks good enough to look like a genuine Netflix show? Then it's "okay". Does the image looks good enough to look like a well-designed illustrations? Then it's "okay". The biggest problem I've seen with generative AI is exactly this "okay"-ness.

But I digress. It's a good thing this is only for super-resolution, with which we only need to care about variety of local texture (it needs to be able to handle all kinds of inputs within our intended domain) instead of overall semantic info. With the dataset ready we move forward with the plan.

The overall plan goes roughly as follows:

• With each image we can obviously obtain a smaller version of that image and an enlarged version of that smaller version (we'll call this one "interpolated"), which will have the same resolution but its details lost.
• We'll pair each interpolated image with its original version and train a neural network that learns how to transform the former into the latter; with this it'll know how to "sharpen" a blurry interpolated image, and when we need to upscale we'll simply enlarge the image first via interpolation and then feed it through the neural network to "sharpen" the details.

I decided to do something small first so that I know it could work:

• The network would be a plain CNN with:
  • a 3 -> 32 -> 64 channel head, an 8-layer 64 -> 64 channel body, and a 64 -> 32 -> 3 channel tail;
    • with a layer of SiLU between all of them.
  • a "global residual connection" which adds the input to the output of the tail.
    • This is recommended by the LLM I use for judging my work (GLM-5.3 Flash), which is the semantically "correct" way because we are training an image sharpening network. Theoretically it could still work without this residual connection (gradient would still be fading out during backpropagation amongst the layers!), it probably would take more training steps and could be more unstable.
• When using this network, the input image would:
  • first be padded by 12 pixels all-around;
    • This is recommended by LLM, because again this is the "semantically more correct" approach, because with 12 layers of 3x3 convolution each pixel requires a surrounding of 25x25 pixels (i.e. 12 on each side). My original plan was to just divide the image into 96x96 patches with some dedicated handling at the right-most and bottom side in case the input image doesn't divide neatly into a grid of 96x96 patches, which I don't think would bring much of a (negative) difference here since the network was trained without a special padding parameter which, with the default configuration of PyTorch's torch.nn.Conv2d being padding by zeroes, means we're not having any "meaningful context" with the edge & corner pixels of a patch anyway. The 12px padding suggestion from LLM was probably made just to make the code cleaner and/or in case we train it with a different kind of padding in those Conv2d's. I would change the structure of the network after this so this is kinda moot.
  • then splitted into 96x96 patches;
  • then the patches would be fed into the model one by one;
  • then we'll stitch the result together.
• We'll train it on my dataset for 10k steps just to see how thing goes.

Other than the reasoning for the residual connection and the pixel padding, all decisions (e.g. how many layers, how many steps, SiLU vs. GELU vs. others) are made randomly and is just me being like "oh I'll do some bullshit and we'll see if it works". Naturally there would be two questions to answer before we can confirm if it actually does work:

• Is this better than enlarging the image w/ interpolation + algorithmically sharpening?
• Is this better than interpolation at all?

To my surprise my 10k step CNN model's answer to both of these questions seem to be "yes". Here's a comparison of enlarging a small picture of Hatsune Miku (courtesy of Piapro; please check here for the licensing), from left to right: interpolated (with PIL.Image.Resampling.BICUBIC), interpolated & sharpened by my network (BILINEAR, which is the one used during training) & me manually sharpen the bicubic interpolated image to my best effort under GIMP:

GIMP's sharpening filter seems to not play well with JPEG compression artifacts and would enhance the "blocky" textures, which my model seems to have smartly smoothed out somewhat. See this close-up comparison of the left pigtail's left-most edge; my model (the middle one) generated a much thinner and much less "staircase"-like line.

For this I consulted LLM and it gave me a few options:

• Data augmentation & train for longer (e.g. 100k)
  • Having more texture variety is almost always a good idea especially if you can get it for free; also my first attempt is just randomly croping a square out of 8 images for 10k times, there's probably going to be certain textures that are present in the dataset but the model has never got a chance to see.
• Use a learning-rate schedule that decreases the learning rate gradually
  • This is to stop the gradient from overshooting and circling around the bottom of the local optimum refusingto go down further towards the end of the learning process
• Use L1 loss instead of L2
  • Apparently the reasoning is that L1 loss would punish pixels that "changes too quick in a local area" (e.g. black lines at the edges of a character's face surrounded by white background) less than L2 because you'd get much smaller gradient values among those kind of places.
• Use ResNet (and a ResNet with more layers) instead of plain CNN.

I did all of those (except for training for 100k steps); but to be quite honest, at 10k steps it's borderline impossible for me to notice any big improvements just by looking; for this reason I decided to just go for 100k. And to be honest - while there definitely is an improvement, I think most of it comes from replacing L2 loss with L1 loss; I don't think any other of the options listed above made too much of a difference.

The code can be found at https://github.com/bctnry/sanime2x and the model can be found at https://huggingface.co/bctnry/sanime2x.

---

Back

Last update: 2026.9.21