タイトル : SINGLE型の切捨てについて 投稿日 : 2016/11/08(Tue) 16:43 投稿者 : SUZUKI
また壁にぶつかってしまいました 下記はSINGLE型にて小数点N位以下を切捨てするロジックです (今回は小数点第4位以下を切り捨て) 型を色々変えていますがうまく行っていません DOUBLE型は色々検索に出て来るのですが SINGLE型はうまくヒットしませんでした? Math.Floor関数がDOUBLE型なので変換誤差をうまく吸収できずにいます Math.Floor関数あきらめたほうがよいか教えていただきたく よろしくお願いします Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click Dim a As Single Dim b As Single a = 1.00401 b = CSng(csp_double_syousuu(CDbl(a), 3, 0)) MsgBox(b) ' 1.004 OK b = CSng(csp_single_syousuu(a, 3, 0)) MsgBox(b) ' 1.004 OK b = CSng(csp_decimal_syousuu(CDec(a), 3, 0)) MsgBox(b) ' 1.004 OK a = 1.004 b = CSng(csp_double_syousuu(CDbl(a), 3, 0)) MsgBox(b) '1.003 NO b = CSng(csp_single_syousuu(a, 3, 0)) MsgBox(b) ' 1.003 NO b = CSng(csp_decimal_syousuu(CDec(a), 3, 0)) MsgBox(b) ' 1.004 OK a = 1.00301 b = CSng(csp_double_syousuu(CDbl(a), 3, 0)) MsgBox(b) ' 1.003 OK b = CSng(csp_single_syousuu(a, 3, 0)) MsgBox(b) ' 1.003 OK b = CSng(csp_decimal_syousuu(CDec(a), 3, 0)) MsgBox(b) ' 1.003 OK a = 1.003 b = CSng(csp_double_syousuu(CDbl(a), 3, 0)) MsgBox(b) '1.003 OK b = CSng(csp_single_syousuu(a, 3, 0)) MsgBox(b) ' 1.003 OK b = CSng(csp_decimal_syousuu(CDec(a), 3, 0)) MsgBox(b) ' 1.002 NO End Sub ''' 見ずらく申し訳ありませんがdValueの型を色々変えています Function csp_single_syousuu(ByRef dValue As Single, ByRef iDigits As Integer, ByRef isw As Integer) As Double Dim dCoef As Double = System.Math.Pow(10, iDigits) Select Case isw Case 0 Return System.Math.Floor(dValue * dCoef) / dCoef End Select End Function Function csp_double_syousuu(ByRef dValue As Double, ByRef iDigits As Integer, ByRef isw As Integer) As Double Dim dCoef As Double = System.Math.Pow(10, iDigits) Select Case isw Case 0 Return System.Math.Floor(dValue * dCoef) / dCoef End Select End Function Function csp_decimal_syousuu(ByRef dValue As Decimal, ByRef iDigits As Integer, ByRef isw As Integer) As Double Dim dCoef As Double = System.Math.Pow(10, iDigits) Select Case isw Case 0 Return System.Math.Floor(dValue * dCoef) / dCoef End Select End Function |