Language…
22 users online:  AmperSam, Anas,  Anorakun, DanMario24YT, GiraffeKiller, Golden Yoshi, Gorry, Green, Green Jerry, h.carrell, Hammerer, Heitor Porfirio, Maw, Metal-Yoshi94, playagmes169, Ray Hamilton, Rykon-V73, Seamus Sinclair, ShoopDaWhoop, Sniwott,  StayAtHomeStegosaurus, Tulip Time Scholarship Games - Guests: 288 - Bots: 571
Users: 64,795 (2,370 active)
Latest user: mathew

Color math and layer priorities problem (solved)

Hi guys.

I'm currently building a level taking place in an building, where some part of the level is seen through a security camera. Therefore, I'd like to use an HDMA gradient to mimic scanlines affecting all layers except 3, as that latter would be in front of everything and show the status bar + a sleeping security guard. Here's the concept screenshot:


The problem is, the closest I've been able to approach that result is this (test level):


(It can't be seen on the screenshot, but the layer 3 guard is behind the layer 2 background, and is affected by the fake scanlines)
As you can see, the status bar section (pre-IRQ) works fine, but the regular section (post-IRQ) always puts layer 3 in subscreen, despite the fact that I check the priority option in LM.
There's also some oddity with sprites: only palettes 4,5,6 and 7 are affected by the fake scanlines, while the others aren't.

So, my questions are:
- How can I solve the layer 3 problem and put layer 3 in front of everything ?
- Is it possible to edit which sprite palettes are affected by my effect and if yes how ?

For reference, this is the code I'm currently using. It's an unmodified output from EffecTool:
Code
init:                    ; Code to be inserted INIT
   LDA #$17    ;\  BG1, BG2, BG3, OBJ on main screen (TM)
   STA $212C   ; | 
   LDA #$00    ; | 0 on main screen should use windowing. (TMW)
   STA $212E   ;/  
   LDA #$00    ;\  0 on sub screen (TS)
   STA $212D   ; | 
   LDA #$00    ; | 0 on sub screen should use windowing. (TSW)
   STA $212F   ;/  
   LDA #$B7    ; BG1, BG2, BG3, OBJ, Backdrop, Half for color math
   STA $40     ;/  mirror of $2131
   
	REP #$20
	LDA #$3202
	STA $4330
	LDA #.RedGreenTable
	STA $4332
	LDY.b #.RedGreenTable>>16
	STY $4334
	LDA #$3200
	STA $4340
	LDA #.BlueTable
	STA $4342
	LDY.b #.BlueTable>>16
	STY $4344
	SEP #$20
	LDA #$18
	TSB $6D9F
   
   RTL
Originally posted by Mathos
- How can I solve the layer 3 problem and put layer 3 in front of everything ?

My closest guess is that the layer 3 tiles you use haven't got any priority because that's one of the three factors (l3 on main screen, higher priority for l3 tiles). Only these with priority goes in front of everything if the other two things are set.

Originally posted by Mathos
- Is it possible to edit which sprite palettes are affected by my effect and if yes how ?

It is hardcoded in the SNES so you can't.
Originally posted by MarioFanGamer
My closest guess is that the layer 3 tiles you use haven't got any priority because that's one of the three factors (l3 on main screen, higher priority for l3 tiles). Only these with priority goes in front of everything if the other two things are set.

Thanks for the tip ! It only solved the priority problem, though: the layer is still affected by the fake scanlines.


Originally posted by MarioFanGamer
Originally posted by Mathos
- Is it possible to edit which sprite palettes are affected by my effect and if yes how ?

It is hardcoded in the SNES so you can't.

Really ? I thought this was some sort of sprite property set by the game itself... too bad, because I would've really appreciated if at least Mario could be affected.
Nice stuff ya got here. Just wondering if you'd release the fixed code for this sometime. Also, how many ExGFX files did the security guard and the table take?
My Mode 0 guide.

My Discord server. It has a lot of archived ASM stuff, so check that out!

Code
   LDA #$B3    ; BG1, BG2, OBJ, Backdrop, Half for color math
   STA $40     ;/  mirror of $2131

If you don't want layer 3 to participate in color math, maybe not enabling it fixes stuff :3
Anime statistic on MyAnimeList:
400 animes completed ✓
6000 episodes completed ✓
100 Days completed ✓
... what even am I doing with my life?
Originally posted by JackTheSpades
Code
   LDA #$B3    ; BG1, BG2, OBJ, Backdrop, Half for color math
   STA $40     ;/  mirror of $2131

If you don't want layer 3 to participate in color math, maybe not enabling it fixes stuff :3

Whoops. I should've double-checked the output. Thanks for the help, it works perfectly now ! :)
That makes me think, having an option for enabling FG gradients on certain layers only would be a nice addition to EffecTool.

Originally posted by MasterSkodwarde
Nice stuff ya got here. Just wondering if you'd release the fixed code for this sometime.

This is for the collab hack I'm leading so no, sorry. Besides, you can recreate it easily now.
Originally posted by MasterSkodwarde
Also, how many ExGFX files did the security guard and the table take?

Just two regular layer 3 ExGFX files.