MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Compute

 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Application Programming
View previous topic :: View next topic  
Author Message
abracadabra
Beginner


Joined: 02 Sep 2003
Posts: 101
Topics: 55
Location: India
PostPosted: Thu Feb 26, 2004 1:46 am    Post subject: Compute Reply with quote
I am facing this strange problem. I have 3 fields
A - 9(2)V9(5)
B - 9(2)V9(5)
C - 9(2)V9(5)

Now B*A gives C.

When I do the calculation of B*A in a calculator or excel it is giving the value desired. Please note here that both B and A's actual values are more than 5 decimal places.

However, when the same calculation takes place in a COBOL program, the value of C is not coming the same as the calculator/excel. This is because the program is first taking till the 6th decimal place and calculating internally and then mving to C rounded

What could be the reason?

Thanks
_________________
Cheers!
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12369
Topics: 75
Location: San Jose
PostPosted: Mon Mar 01, 2004 2:50 pm    Post subject: Reply with quote
Abracadabra,

Check this link for Intermediate results and arithmetic precision


Hope this helps...

Cheers

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Application Programming All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group