behavior of division?

More
4 years 9 months ago - 4 years 9 months ago #1 by kingeric1992
behavior of division? was created by kingeric1992
What is the behavior of division "/" ?

0.03125 != 1/32 == 0 on my test.

for some reason,
float4 threshold = float4(index.x, index.x + 1/64, index.x + 1/32, index.x + 3/64);
and
float4 threshold = float4(index.x, index.x + 0.015625, index.x + 0.03125, index.x + 0.046875);
are having different result.
Last edit: 4 years 9 months ago by kingeric1992.

Please Log in or Create an account to join the conversation.

More
4 years 9 months ago #2 by crosire
Replied by crosire on topic behavior of division?
That's because "1" and "32" are integers. A divison with two integers results in an integer again (in your case zero). To fix this, at least one of the factors has to be a float, i.e. "1.0/32".

Please Log in or Create an account to join the conversation.

More
4 years 9 months ago #3 by kingeric1992
Replied by kingeric1992 on topic behavior of division?
Thanks.

Please Log in or Create an account to join the conversation.