behavior of division?

  • Posts: 177
4 years 1 month ago - 4 years 1 month ago #1 by kingeric1992
What is the behavior of division "/" ?

0.03125 != 1/32 == 0 on my test.

for some reason,
float4 threshold = float4(index.x, index.x + 1/64, index.x + 1/32, index.x + 3/64);
and
float4 threshold = float4(index.x, index.x + 0.015625, index.x + 0.03125, index.x + 0.046875);
are having different result.

Please Log in or Create an account to join the conversation.

  • Posts: 3669
4 years 1 month ago #2 by crosire
That's because "1" and "32" are integers. A divison with two integers results in an integer again (in your case zero). To fix this, at least one of the factors has to be a float, i.e. "1.0/32".

Please Log in or Create an account to join the conversation.

  • Posts: 177
4 years 1 month ago #3 by kingeric1992
Thanks.

Please Log in or Create an account to join the conversation.