Bh8 and Rg4?
"Just make it a triple check..."

Yes, that's correct. The triple check is illegal, but removing any non-King unit leaves a double check that is easily legal (all discovered checks, naturally).

Here's another illegal cluster I made a long time ago.
Add a 1 black queen, 1 black rook, 1 black bishop, 1 black knight and 1 black pawn to form an illegal cluster (2 solutions):

Do you really mean that the task is only to put the pieces on the board, without further conditions?

No, what it is is that the position is illegal, but if you remove one non-king piece then it becomes legal.

Here's another illegal cluster I made a long time ago.
Add a 1 black queen, 1 black rook, 1 black bishop, 1 black knight and 1 black pawn to form an illegal cluster (2 solutions):
add pawn to any of the 1st or 8th rank files and the position is illegal anyway. there should be atleast 1000 solutions for this

add pawn to any of the 1st or 8th rank files and the position is illegal anyway. there should be atleast 1000 solutions for this
The position is illegal, but if you remove any piece, it must become legal. In your case, if you remove a non-pawn piece, the position remains illegal, so that's wrong.

Technically, the position is legal. It only works in the desired way if you state "Black to move". It's a nice construction regardless, of which I suspect there are multiple variations.
The intended solutions have no additional conditions. The position itself suffices.

If it is black to move what do you suggest was whites last move?
If it is white to move what was the last move by black and what did white play before that?

I just realised I failed miserably. If blacks last move was a capture, then white could have moved the piece that was captured before that. My solution would only work if whites last move was a king move.

You are right. For some reason I thought 2 pieces covering d4 meant the king could not have moved from there.
Illegal Cluster. Add 1 Black bishop and 1 Black rook to the diagram to form an illegal cluster: a position that is illegal, but when you remove any non-King unit, it becomes legal.
T. R. Dawson
The Problemist Fairy Suppl., 1933