CAST和IsNumeric

为什么下面的查询返回“错误转换数据typesvarchar到bigint”? 不是IsNumeric使CAST安全吗? 我已经尝试了每个数字数据types的转换,并得到相同的“错误转换…”错误。 我不相信结果数字的大小是一个问题,因为溢出是一个不同的错误。

有趣的是,在pipe理工作室中,结果实际上显示在结果窗格中一秒钟之后,错误又回来了。

SELECT CAST(myVarcharColumn AS bigint) FROM myTable WHERE IsNumeric(myVarcharColumn) = 1 AND myVarcharColumn IS NOT NULL GROUP BY myVarcharColumn 

有什么想法吗?

如果varchar值可以转换为任何数字types,则IsNumeric返回1。 这包括int,bigint,decimal,numeric,real&float。

科学记数法可能会导致你的问题。 例如:

 Declare @Temp Table(Data VarChar(20)) Insert Into @Temp Values(NULL) Insert Into @Temp Values('1') Insert Into @Temp Values('1e4') Insert Into @Temp Values('Not a number') Select Cast(Data as bigint) From @Temp Where IsNumeric(Data) = 1 And Data Is Not NULL 

有一个技巧可以和IsNumeric一起使用,以便用科学记数法返回数字0。 您可以应用类似的技巧来防止十进制值。

IsNumeric(YourColumn +'e0')

IsNumeric(YourColumn +'.0e0')

试试看。

 SELECT CAST(myVarcharColumn AS bigint) FROM myTable WHERE IsNumeric(myVarcharColumn + '.0e0') = 1 AND myVarcharColumn IS NOT NULL GROUP BY myVarcharColumn 

背景:

我使用第三方数据库,不断接收来自其他第三方供应商的新数据。
parsing出用于存储结果的可怕varchar字段是我的工作。
我们希望parsing出尽可能多的数据,这个解决scheme向您展示了如何“清理”数据,使有效的条目不被忽略。

  1. 一些结果是免费发短信。
  2. 一些枚举(是,不,蓝色,黑色等)。
  3. 有些是整数。
  4. 其他人使用小数。
  5. 许多是百分比,如果转换为整数,可能会让你后来启动。

如果我需要查询一个给定的小数范围(如适用-1.4到3.6),我的选项是有限的。
我在下面更新了我的查询,使用@GMastrosbuild议附加'e0'。
感谢@Gasastros,这为我节省了额外的2行逻辑。

解:

 --NOTE: I'd recommend you use this to convert your numbers and store them in a separate table (or field). -- This way you may reuse them when when working with legacy/3rd-party systems, instead of running these calculations on the fly each time. SELECT Result.Type, Result.Value, Parsed.CleanValue, Converted.Number[Number - Decimal(38,4)], (CASE WHEN Result.Value IN ('0', '1', 'True', 'False') THEN CAST(Result.Value as Bit) ELSE NULL END)[Bit],--Cannot convert 1.0 to Bit, it must be in Integer format already. (CASE WHEN Converted.Number BETWEEN 0 AND 255 THEN CAST(Converted.Number as TinyInt) ELSE NULL END)[TinyInt], (CASE WHEN Converted.Number BETWEEN -32768 AND 32767 AND Result.Value LIKE '%\%%' ESCAPE '\' THEN CAST(Converted.Number / 100.0 as Decimal(9,4)) ELSE NULL END)[Percent], (CASE WHEN Converted.Number BETWEEN -32768 AND 32767 THEN CAST(Converted.Number as SmallInt) ELSE NULL END)[SmallInt], (CASE WHEN Converted.Number BETWEEN -214748.3648 AND 214748.3647 THEN CAST(Converted.Number as SmallMoney) ELSE NULL END)[SmallMoney], (CASE WHEN Converted.Number BETWEEN -2147483648 AND 2147483647 THEN CAST(Converted.Number as Int) ELSE NULL END)[Int], (CASE WHEN Converted.Number BETWEEN -2147483648 AND 2147483647 THEN CAST(CAST(Converted.Number as Decimal(10)) as Int) ELSE NULL END)[RoundInt],--Round Up or Down instead of Truncate. (CASE WHEN Converted.Number BETWEEN -922337203685477.5808 AND 922337203685477.5807 THEN CAST(Converted.Number as Money) ELSE NULL END)[Money], (CASE WHEN Converted.Number BETWEEN -9223372036854775808 AND 9223372036854775807 THEN CAST(Converted.Number as BigInt) ELSE NULL END)[BigInt], (CASE WHEN Parsed.CleanValue IN ('1', 'True', 'Yes', 'Y', 'Positive', 'Normal') THEN CAST(1 as Bit) WHEN Parsed.CleanValue IN ('0', 'False', 'No', 'N', 'Negative', 'Abnormal') THEN CAST(0 as Bit) ELSE NULL END)[Enum], --I couln't use just Parsed.CleanValue LIKE '%e%' here because that would match on "True" and "Negative", so I also had to match on only allowable characters. - 02/13/2014 - MCR. (CASE WHEN ISNUMERIC(Parsed.CleanValue) = 1 AND Parsed.CleanValue LIKE '%e%' THEN Parsed.CleanValue ELSE NULL END)[Exponent] FROM ( VALUES ('Null', NULL), ('EmptyString', ''), ('Spaces', ' - 2 . 8 % '),--Tabs and spaces mess up IsNumeric(). ('Bit', '0'), ('TinyInt', '123'), ('Int', '123456789'), ('BigInt', '1234567890123456'), --('VeryLong', '12345678901234567890.1234567890'), ('VeryBig', '-1234567890123456789012345678901234.5678'), ('TooBig', '-12345678901234567890123456789012345678.'),--34 (38-4) is the Longest length of an Integer supported by this query. ('VeryLong', '-1.2345678901234567890123456789012345678'), ('TooLong', '-12345678901234567890.1234567890123456789'),--38 Digits is the Longest length of a Number supported by the Decimal data type. ('VeryLong', '000000000000000000000000000000000000001.0000000000000000000000000000000000000'),--Works because Casting ignores leading zeroes. ('TooLong', '.000000000000000000000000000000000000000'),--Exceeds the 38 Digit limit for all Decimal types after the decimal-point. --Dot(.), Plus(+), Minus(-), Comma(,), DollarSign($), BackSlash(\), Tab(0x09), and Letter-E(e) all yeild false-posotives with IsNumeric(). ('Decimal', '.'), ('Decimal', '.0'), ('Decimal', '3.99'), ('Positive', '+'), ('Positive', '+20'), ('Negative', '-'), ('Negative', '-45'), ('Negative', '- 1.23'), ('Comma', ','), ('Comma', '1,000'), ('Money', '$'), ('Money', '$10'), ('Percent', '%'), ('Percent', '110%'),--IsNumeric will kick out Percent(%) signs. ('BkSlash', '\'), ('Tab', CHAR(0x09)),--I've actually seen tab characters in our data. ('Exponent', 'e0'), ('Exponent', '100e-999'),--No SQL-Server datatype could hold this number, though it is real. ('Enum', 'True'), ('Enum', 'Negative') ) AS Result(Type, Value)--O is for Observation. CROSS APPLY ( --This Step is Optional. If you have Very Long numbers with tons of leading zeros, then this is useful. Otherwise is overkill if all the numbers you want have 38 or less digits. --Casting of trailing zeros count towards the max 38 digits Decimal can handle, yet Cast ignores leading-zeros. This also cleans up leading/trailing spaces. - 02/25/2014 - MCR. SELECT LTRIM(RTRIM(SUBSTRING(Result.Value, PATINDEX('%[^0]%', Result.Value + '.'), LEN(Result.Value))))[Value] ) AS Trimmed CROSS APPLY ( SELECT --You will need to filter out other Non-Keyboard ASCII characters (before Space(0x20) and after Lower-Case-z(0x7A)) if you still want them to be Cast as Numbers. - 02/15/2014 - MCR. REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(Trimmed.Value,--LTRIM(RTRIM(Result.Value)), (CHAR(0x0D) + CHAR(0x0A)), ''),--Believe it or not, we have people that press carriage return after entering in the value. CHAR(0x09), ''),--Apparently, as people tab through controls on a page, some of them inadvertently entered Tab's for values. ' ', ''),--By replacing spaces for values (like '- 2' to work), you open the door to values like '00 12 3' - your choice. '$', ''), ',', ''), '+', ''), '%', ''), '/', '')[CleanValue] ) AS Parsed--P is for Parsed. CROSS APPLY ( --NOTE: I do not like my Cross-Applies to feed into each other. -- I'm paranoid it might affect performance, but you may move this into the select above if you like. - 02/13/2014 - MCR. SELECT (CASE WHEN ISNUMERIC(Parsed.CleanValue + 'e0') = 1--By concatenating 'e0', I do not need to check for: Parsed.CleanValue NOT LIKE '%e%' AND Parsed.CleanValue NOT IN ('.', '-') -- If you never plan to work with big numbers, then could use Decimal(19,4) would be best as it only uses 9 storage bytes compared to the 17 bytes that 38 precision requires. -- This might help with performance, especially when converting a lot of data. AND CHARINDEX('.', REPLACE(Parsed.CleanValue, '-', '')) - 1 <= (38-4)--This is the Longest Integer supported by Decimal(38,4)). AND LEN(REPLACE(REPLACE(Parsed.CleanValue, '-', ''), '.', '')) <= 38--When casting to a Decimal (of any Precision) you cannot exceed 38 Digits. - 02/13/2014 - MCR. THEN CAST(Parsed.CleanValue as Decimal(38,4))--Scale of 4 used is the max that Money has. This is the biggest number SQL Server can hold. ELSE NULL END)[Number] ) AS Converted--C is for Converted. 

输出:

下面的屏幕截图被格式化并剪切,以适应StackOverflow。
实际的结果有更多的专栏。 MikeTeeVee的IsNumeric Casting

研究:

每个查询的旁边是结果。
看到IsNumeric的缺点以及CASTing的局限性很有意思。
我展示了这一点,以便您可以看到上面写入查询的背景研究。
理解每一个devise决定是很重要的(如果你正在考虑削减任何东西)。

 SELECT ISNUMERIC('')--0. This is understandable, but your logic may want to default these to zero. SELECT ISNUMERIC(' ')--0. This is understandable, but your logic may want to default these to zero. SELECT ISNUMERIC('%')--0. SELECT ISNUMERIC('1%')--0. SELECT ISNUMERIC('e')--0. SELECT ISNUMERIC(' ')--1. --Tab. SELECT ISNUMERIC(CHAR(0x09))--1. --Tab. SELECT ISNUMERIC(',')--1. SELECT ISNUMERIC('.')--1. SELECT ISNUMERIC('-')--1. SELECT ISNUMERIC('+')--1. SELECT ISNUMERIC('$')--1. SELECT ISNUMERIC('\')--1. ' SELECT ISNUMERIC('e0')--1. SELECT ISNUMERIC('100e-999')--1. No SQL-Server datatype could hold this number, though it is real. SELECT ISNUMERIC('3000000000')--1. This is bigger than what an Int could hold, so code for these too. SELECT ISNUMERIC('1234567890123456789012345678901234567890')--1. Note: This is larger than what the biggest Decimal(38) can hold. SELECT ISNUMERIC('- 1')--1. SELECT ISNUMERIC(' 1 ')--1. SELECT ISNUMERIC('True')--0. SELECT ISNUMERIC('1/2')--0. No love for fractions. SELECT CAST('e0' as Int)--0. Surpise! Casting to Decimal errors, but for Int is gives us zero, which is wrong. SELECT CAST('0e0' as Int)--0. Surpise! Casting to Decimal errors, but for Int is gives us zero, which is wrong. SELECT CAST(CHAR(0x09) as Decimal(12,2))--Error converting data type varchar to numeric. --Tab. SELECT CAST(' 1' as Decimal(12,2))--Error converting data type varchar to numeric. --Tab. SELECT CAST(REPLACE(' 1', CHAR(0x09), '') as Decimal(12,2))--Error converting data type varchar to numeric. --Tab. SELECT CAST('' as Decimal(12,2))--Error converting data type varchar to numeric. SELECT CAST('' as Int)--0. Surpise! Casting to Decimal errors, but for Int is gives us zero, which is wrong. SELECT CAST(',' as Decimal(12,2))--Error converting data type varchar to numeric. SELECT CAST('.' as Decimal(12,2))--Error converting data type varchar to numeric. SELECT CAST('-' as Decimal(12,2))--Arithmetic overflow error converting varchar to data type numeric. SELECT CAST('+' as Decimal(12,2))--Arithmetic overflow error converting varchar to data type numeric. SELECT CAST('$' as Decimal(12,2))--Error converting data type varchar to numeric. SELECT CAST('$1' as Decimal(12,2))--Error converting data type varchar to numeric. SELECT CAST('1,000' as Decimal(12,2))--Error converting data type varchar to numeric. SELECT CAST('- 1' as Decimal(12,2))--Error converting data type varchar to numeric. (Due to spaces). SELECT CAST(' 1 ' as Decimal(12,2))--1.00 Leading and trailing spaces are okay. SELECT CAST('1.' as Decimal(12,2))--1.00 SELECT CAST('.1' as Decimal(12,2))--0.10 SELECT CAST('-1' as Decimal(12,2))--1.00 SELECT CAST('+1' as Decimal(12,2))--1.00 SELECT CAST('True' as Bit)--1 SELECT CAST('False' as Bit)--0 --Proof: The Casting to Decimal cannot exceed 38 Digits, even if the precision is well below 38. SELECT CAST('1234.5678901234567890123456789012345678' as Decimal(8,4))--1234.5679 SELECT CAST('1234.56789012345678901234567890123456789' as Decimal(8,4))--Arithmetic overflow error converting varchar to data type numeric. --Proof: Casting of trailing zeros count towards the max 38 digits Decimal can handle, yet it ignores leading-zeros. SELECT CAST('.00000000000000000000000000000000000000' as Decimal(8,4))--0.0000 --38 Digits after the decimal point. SELECT CAST('000.00000000000000000000000000000000000000' as Decimal(8,4))--0.0000 --38 Digits after the decimal point and 3 zeros before the decimal point. SELECT CAST('.000000000000000000000000000000000000000' as Decimal(8,4))--Arithmetic overflow error converting varchar to data type numeric. --39 Digits after the decimal point. SELECT CAST('1.00000000000000000000000000000000000000' as Decimal(8,4))--Arithmetic overflow error converting varchar to data type numeric. --38 Digits after the decimal point and 1 non-zero before the decimal point. SELECT CAST('000000000000000000000000000000000000001.0000000000000000000000000000000000000' as Decimal(8,4))--1.0000 --Caveats: When casting to an Integer: SELECT CAST('3.0' as Int)--Conversion failed when converting the varchar value '3.0' to data type int. --NOTE: When converting from character data to Int, you may want to do a double-conversion like so (if you want to Round your results first): SELECT CAST(CAST('3.5' as Decimal(10)) as Int)--4. Decimal(10) has no decimal precision, so it rounds it to 4 for us BEFORE converting to an Int. SELECT CAST(CAST('3.5' as Decimal(11,1)) as Int)--3. Decimal (11,1) HAS decimal precision, so it stays 3.5 before converting to an Int, which then truncates it. --These are the best ways to go if you simply want to Truncate or Round. SELECT CAST(CAST('3.99' as Decimal(10)) as Int)--3. Good Example of Rounding. SELECT CAST(FLOOR('3.99') as Int)--3. Good Example fo Truncating. 

最好的解决scheme是停止将整数存储在varchar列中。 显然有一个数据问题,数据可以解释为一个数字,但不能像这样。 你需要find问题的logging,如果数据是可以修改的,那么就修复它们。 根据你正在存储的内容和为什么它是一个varchar开始,你可能需要修复查询而不是数据。 但是,如果您首先find炸毁当前查询的logging,那么这样做也会更容易。

如何做到这一点是问题。 在数据中search小数点位置比较容易,看看是否有使用charindex的小数(除了会转换为0)。 您还可以查找任何包含e或$的logging,或者根据已经给出的来源查找可能被数字化的任何其他字符。 如果你没有很多的logging,那么对数据进行快速的可视化扫描可能会发现它,特别是如果你先sorting的话。

有时,当我一直在发现查询的不良数据时,我把数据放到一个临时表中,然后尝试批量处理(使用插值),直到find它发生的那个数据为止。 从第一个1000开始(不要忘记使用order by,否则删除好的logging时不会得到相同的结果,如果有数以百万计的logging以较大的数字开始,1000只是最好的猜测)。 如果通过,则删除这1000条logging并select下一批。 一旦失败,请select一个较小的批次。 一旦你的数字很容易被视觉扫描,你会发现这个问题。 当我有几百万条logging和一个奇怪的错误,我已经能够很快地find问题logging,我没有试过任何查询(这基本上是猜测什么可能是错误的)已经find了问题。

试试这个,看看你是否仍然有错误…

 SELECT CAST(CASE WHEN IsNumeric(myVarcharColumn) = 0 THEN 0 ELSE myVarcharColumn END AS BIGINT) FROM myTable WHERE IsNumeric(myVarcharColumn) = 1 AND myVarcharColumn IS NOT NULL GROUP BY myVarcharColumn 

尝试包装它的情况下:

 select CASE WHEN IsNumeric(mycolumn) = 1 THEN CAST(mycolumn as bigint) END FROM stack_table WHERE IsNumeric(mycolumn) = 1 GROUP BY mycolumn 

根据BOL ISNUMERIC在inputexpression式求值为有效的数字数据types时返回1; 否则返回0。

有效的数字数据types包括以下内容:

  • INT
  • 数字
  • BIGINT
  • SMALLINT
  • SMALLMONEY
  • TINYINT
  • 浮动
  • 十进制
  • 真实

正如其他人指出,你将有一些数据将通过ISNUMERICtesting,但无法投射到bigint

ISNUMERIC只是…愚蠢的。 你完全可以使用它。 所有情况下返回1:

 ISNUMERIC('-') ISNUMERIC('.') ISNUMERIC('-$.') 

对于任何整数types,而不是使用: ISNUMERIC(@Value) = 1只使用: (@Value NOT LIKE '[^0-9]') OR (@Value NOT LIKE '-[^0-9]'

唯一的好方法是不要使用ISNUMERIC。

我有同样的问题,我想到了2008年SQL的标量函数

 ALTER Function [dbo].[IsInteger](@Value VarChar(18)) Returns Bit As Begin Return IsNull( (Select Case When CharIndex('.', @Value) > 0 Then 0 Else 1 End Where IsNumeric(@Value + 'e0') = 1), 0) End 

如果你在2012年,你可以使用TRY_CONVERT

我在MSSQL 2014中触发相同的问题,而不是句号:isnumeric('9090,23')给出1; 投(“9090,23”为浮动)失败

我用''replace','。

有DAXfunction(IsError或IfError)可以帮助在这种情况下,但我们没有在我们的SQL Server 2008 R2的。 看起来像SQL Server的一些额外的分析包。

我遇到这个博客文章,可能有帮助。 我以前没有遇到过这个问题,不确定在这个例子中是否能帮到你。

http://dotmad.blogspot.com/2007/02/cannot-call-methods-on-bigint-error.html