Handling a datetime in NAV with super accuracy

So, as I have promised – some neater code for exact datetime handling in NAV.

The same concept, as described in the earlier post. I am going to use text string to store formatted and sortable datetime value. The only difference – I’m employing a power of .NET to do so. .NET enables me to utilise culture, and most importantly CultureInfo.InvariantCulture. Why would I do so? Simply because initial root of issue was the fact that I was in need to accept by NAV a very accurate datetime from external sources. I can hardly imagine a situation when I would need exact datetime in NAV. Dealing with data from other app which are much more accurate in terms of time handling comparing to NAV perhaps is only case when we need to deal with milliseconds.

So, then the goal is to store data within NAV SQL database with high level of accuracy and same time to allow external system to read and write this data.

And this is simple: two new functions – FormatDT  and EvaluateDT – and this is pretty much it.

The first one, FormatDT simply accept datetime value. I am assuming my datetime come from webservice, therefore always UTC. So I just apply to it CultureInfo.InvariantCulture and convert it to string. Therefore, it saved in SQL DB in UTC, but as string. Same as NAV handle dates (yes, dates are always UTC in NAV SQL DB and only formatted to local time on UI). So then in DB we have an UTC date string formatted as sortable datetime. Like this one: ‘2017-08-31T19:58:29.2780000Z’

FormatDT(DateTimeString : DateTime) : Text
DotNetDateTime := DateTimeString;
EXIT(DotNetDateTime.ToString('o', CultureInfo.InvariantCulture));

The next task then is to evaluate it back to datetime to present it on my SOAP service.

The advantage of .NET here is the fact that TryParse method will inherit a parent culture. So, on SOAP service it will always be UTC, as NAV SOAP services have invariant culture.

On NAV UI (windows client, web client, phone, tablet, etc..) this function will inherit the client culture, which is in turn inherited from local machine. Pretty, isn’t it?

EvaluateDT(DateTimeText : Text) : DateTime
DotNetDateTime := DotNetDateTime.DateTime(0);
DotNetDateTime.TryParse(DateTimeText, DotNetDateTime);
EXIT(DotNetDateTime);

So conclusion then:

  1. Datetime stored as text string allowing us a microseconds accuracy, micro, not even milliseconds and stored in SQL DB in UTC, same way as standard NAV would store dates/times in UTC.
  2. Datetime evaluated for UI or SOAP service same way as standard NAV would do.

Job done.

A test objects here:

 

Some notes on MS SQL timestamps in Dynamics NAV 2016.

My data have a timestamp. A datetime datatype. And it is vital to me as business process dictates so some logic decisions must be made based on record datetime.  What I have noticed is that all my datetimes ends with 0 or 3 or 7 milliseconds. Like 01/01/2017 20:54:253. Or 01/01/2017 20:54:250. Or 01/01/2017 20:54:257. But never 01/01/2017 20:54:254.

Quite strange, as I have assumed the tick is a millisecond. Which is true in NAV – the smallest tick is a millisecond (1/1000 second) indeed. So why then always 0 3 or 7 in the database? I quickly created a nav code like this:

OnRun()
MyDatetime := CURRENTDATETIME;
FOR i := 1 TO 1000 DO BEGIN
  CLEAR(MyTable);
  MyTable."Create Date" := MyDatetime + i;
  MyTable.INSERT;
END;

What would you expect of this code? I was expecting 1000 records created with one tick datetime difference. I’m using CLEAR(MyTable); as my table has autoincrement primary key, so MyTable.INIT would not work as INIT does not clearing the primary key value, so INSERT will fail then as we let SQL server to decide primary key.

And indeed, I had 1000 records. And all of them ends with 0, 3 or 7. Mystery? There must be no mystery in IT. So I amended the code then to dump data to simple text file same time as I am inserting to table.

MyFile.TEXTMODE := TRUE;
MyFile.WRITEMODE := TRUE;
MyFile.CREATE('c:\temp\test.txt');
MyDatetime := CURRENTDATETIME;
FOR i := 1 TO 1000 DO BEGIN
  CLEAR(MyTable);
  MyTable."Create Date" := MyDatetime + i;
  MyTable.INSERT;
  MyFile.WRITE(FORMAT(MyTable."Create Date", 0, '<Day,2>/<Month,2>/<Year4> <Hours24,2>:<Minutes,2>:<Seconds,2><Second dec.><Comma,.>'));
END;
MyFile.CLOSE;

And results were truly surprising.

Text File SQL table
30/08/2017 11:58:29.271 30/08/2017 11:58:29.270
30/08/2017 11:58:29.272 30/08/2017 11:58:29.273
30/08/2017 11:58:29.273 30/08/2017 11:58:29.273
30/08/2017 11:58:29.274 30/08/2017 11:58:29.273
30/08/2017 11:58:29.275 30/08/2017 11:58:29.277
30/08/2017 11:58:29.276 30/08/2017 11:58:29.277
30/08/2017 11:58:29.277 30/08/2017 11:58:29.277
30/08/2017 11:58:29.278 30/08/2017 11:58:29.277
30/08/2017 11:58:29.279 30/08/2017 11:58:29.280
30/08/2017 11:58:29.280 30/08/2017 11:58:29.280

 

As you can see – in the file it is one tick 1/1000 of second, as expected. But in SQL the tick is about 1/300 second and rounded to nearest. Thank you, MS, for your infinite wisdom!

I can prove the point – https://stackoverflow.com/questions/715432/why-is-sql-server-losing-a-millisecond

Of course, there is a SQL datatype datetime2 with precision of 100 nanoseconds. But it is not available in NAV.

So, then conclusion: never use NAV datetime if you want one tick precision. It is not here. Yes, NAV can handle 1/1000 of a second, but SQL cannot!

To prove the point, I have added two new fields to my table: Create Date and Create Time, with NAV datatypes Date and Time respectively. And slightly amended the code

MyDatetime := CURRENTDATETIME;
FOR i := 1 TO 1000 DO BEGIN
  CLEAR(MyTable);
  MyDatetime += 1;
  MyTable."Create DateTime" := MyDatetime;
  MyTable."Create Date" := DT2DATE(MyDatetime);
  MyTable."Create Time" := DT2TIME(MyDatetime);
  MyTable.INSERT;
END;

I somehow was under impression that on SQL end NAV Time datatype handled differently. But it is not. On SQL end NAV Time = SQL datatype DateTime. Which is rounded to 1/300 of a second.

So, there is no way in NAV 2016, 2017 or any other version of NAV to record a time with precision of one tick, I mean 1/1000 second. No way.

Well, there is. A text string of length 23 and some stupid weird code to format datetime received from external sources (where it is high-precision) to string, and back to datetime when our external source read nav data via webservice. I know, it is ugly. But I cannot see better way.

DatetimeToString(MyDatetime : DateTime) : Text
IF MyDatetime = 0DT THEN
  MyDatetime := CREATEDATETIME(01011753D, 000000T);
String := FORMAT(MyDatetime, 0, '<Day,2>/<Month,2>/<Year4> <Hours24,2>:<Minutes,2>:<Seconds,2><Second dec.><Comma,.>');
IF STRLEN(String) = 19 THEN
  String := String + '.';
IF STRLEN(String) < 23 THEN
  String := PADSTR(String, 23, '0');
EXIT(String);
StringToDatetime(MyDatetimeAsString : Text) : DateTime
EVALUATE(Day, COPYSTR(MyDatetimeAsString, 1, 2));
EVALUATE(Month, COPYSTR(MyDatetimeAsString, 4, 2));
EVALUATE(Year, COPYSTR(MyDatetimeAsString, 7, 4));
EVALUATE(MyTime, COPYSTR(MyDatetimeAsString, 12, 12));
MyDate := DMY2DATE(Day, Month, Year);
MyDatetime := CREATEDATETIME(MyDate, MyTime);
EXIT(MyDatetime);

And there is another issue – the datetime stored in NAV always stored in UTC and then formatted to local time on NAV UI. On webservice it is UTC though. So, then all this handling needs to be implemented in your code when you do format datetime to string and vice versa, or to be precise, when you display this data in NAV.

I will cover this in separate post though.

UPDATE: I actually see a better way of managing this. Still using strings, but utilizing the power of .NET – so cleaner and faster code then. New post will follow on this.

Dynamics NAV 2015 and Windows 10

//UPDATE: MS promised to fix everything in October hotfix! http://blogs.msdn.com/b/nav/archive/2015/08/20/windows-10-and-dynamics-nav.aspx

Also I need to add that if you install NAV on windows 8.1 and then upgrade to windows 10 – everything works, including web server components and help server. Fails on fresh install, though!

//UPDATE END

So, as you can see it works! Windows 10 build 10162 and NAV2015 build 8.0.37874.0.

NAV2015onWindows10

Yeah, NAV2015 windows client and server can be installed and run on Windows 10. But, unfortunately not everything works. I am struggling to make live Help Server and Web Server for NAV 2015, it seems to me that MS have done something to IIS in Windows 10, so it is not compatible to NAV 2015 anymore. The error I am getting is the following:

Microsoft Dynamics NAV Build 37874
Error Report
Setup Components
The component was rolled back.
Microsoft IIS URL Rewrite Module
Microsoft IIS URL Rewrite Module (64-bit)
Fatal error during installation.

Microsoft IIS URL Rewrite Module Error

No idea what it is, tried to google, but not a big success.

Anyways, I’ll report this to MS and we will see what will happens in future builds.

 

 

Timestamp and NAV 2015

Doing an integration work I’ve created a new table. I’ve decided to keep filed names as they come in inbound XML file, just to keep things simple. Ok, it is against the MS naming conventions, but no people will look into this intermediate table, so i decide to go this way.

And know what, I’ve get trapped almost immediately. My inbound XML file had a field named ‘timestamp’.

Guess what I got?

This one:

timestamp

Ok, it is clear that any SQL table (and NAV tables are SQL tables) already have column named ‘timestamp’

So, stick to MS naming conventions and you won’t be in trouble :).

Record Link and Notification in NAV 2013R2.

Yeah, no posts for a long time. The reason is simple. I am currently working for a customer on 2009SP1, classic. There is nothing to post about…

But, earlier this year I have had a customer request, when customer wanted some notifications to be created, triggered by NAS. Ok, sound simple, say the condition met and all we need is to insert a text into ‘Note’ field of a ‘Record Link’ table.

There is even a standard code to do so (can be found in the codeunit 454 ‘Job Queue – Send Notification’, local function ‘SetText’. What can be easier?

All coding has been done in little time, matter of hours, but initial testing revealed that the notification text is truncated. Even more, some messages were all right, but some badly truncated…

I’ve tried everything. But, the answer was simple – wrong build. I was on 7.1.36366.0. I’ve upgraded my installation to 7.1.40076.0 – and problem gone. All my truncated notes now looks just fine.

Another reason to keep everything updated!

 

NAV 2013, NAV 2015 RDLC Reports – Font size issue.

As you probably know, it is fairly easy to use HTML formatting on RDLC report controls. Sometimes it is quite useful, especially for things like footers or so, when you need to put something like company details in strictly formatted way, using different fonts and colors for single line of text, like: ‘NAV Workshop LTD Registered address: XXXXXXXX, ZZZZZZZZZ, Warwickshire, United Kingdom’, Company No: 900000′.  Of course, this can be accomplished by creating a separate text box for each formatting, but it is a bit of overkill. HTML formatting like

<span style="font-size: 8pt;"><span style="color: #003366;"><strong>NAV Workshop LTD</strong></span> Registered address</span>: <span style="font-size: 8pt;"><strong>XXXXXXXX, ZZZZZZZZZ, Warwickshire</strong>,</span> <span style="font-size: 8pt;">United Kingdom, Company No:<strong> 900000. </strong></span>

will easily do a trick.

Where the issue then? In code example above there is no issues, but if you use something like

<span style="font-size: Medium"><span style="color: #003366;"><strong>NAV Workshop LTD</strong></span>

– then you re in trouble. Although it is fine to use relative font-size on HTML, but just don’t do it on RDLC reports HTML, as it will be rendered differently on different machines. Just use strict font-size.

Payment Terms, Payment Method and Payment Discounts

In Dynamics NAV 2015 (and in older versions too) we do have a functionality to define payment terms and payment methods. The difference between two is that Payment Terms define WHEN we want money in, and Payment Method define HOW we want money in.
Pretty simple, and it is hard to find a NAV Consultant, who is unfamiliar with this.

But, I’ve managed to find a flaw in this well-known and reliable stuff. Say you define a Payment Terms of 1M(8D) which will mean, that we want money in in 1 month, but will grant an payment discount if customer pay within 8 days. Fine till this point – quite realistic scenario. Then we define a number of payment methods, including CASH, and for cash we put a Bal. Account. This is fine too – when you post an invoice system will automatically post an balancing entry, indicating that payment is received. Exactly what you expect when you take a cash and put it to your cashbox :).

But what if customer’s payment terms defaulted to earlier mentioned 1M(8D)? Well, here we got an issue. The system will automatically grant a payment discount, as you’ve just posted a payment transaction with same date as on your invoice. So in result we will have a payment discount granted, and in all our ledgers the payment amount will be reduced by discount amount, and system will say no single word about this until you navigate on posted document and check you ledgers…

In my opinion this behaviour is entirely wrong for cash based transactions. Well, I know, the payment discounts isn’t a very popular stuff, but as long it is here it worth to check if you customer got payment discount, and just disable it if you going to take cash.

Backup, Restore and Record Link – NAV 2009 R2

Yeah, i know, 2009 is an old one, but it still alive, despite not supported by Microsoft.

Well, we still have a customer, who run NAV 2009 R2, and colleague of mine came across interesting issue: customer reported they lost all records in Record Link table. After some investigation it turned out that they had intention to create a test company – which lead they to the problem.

So, what they did? They took a native NAV backup, then renamed existing company, then restored a backup to bring live company back. Boom! No Record Links anymore!

I am not surprised – Record Link table has ‘data per company = no’ and a primary key ‘Link ID’ .  ‘Company’ field with relation to ‘company.name’ isn’t part of key…

When you made a backup and then rename company ‘A’ to company ‘B’ system will update only ‘company‘ field in the record link table. Then when you restore original company ‘A’ backup if you include data common to all companies – system will not overwrite Record Link table – it will just skip records with same ID (and PK isn’t changed, you see!) .

So, how to solve then? The answer is simple: just a line of code:
RecordLink.Modyfyall(Company, ‘A’) 
assuming there was only one company before.

Yeah, this solve the problem, but what we can do to avoid the issue when we want a test company in old good NAV 2009 R2? My common sense tell me that there is a simple way to avoid that sort of issues.

1. Make a company A backup
2. Restore to clean DB
3. Rename restored to new name (B)
4. Take a backup
5. Restore taken backup to live DB, which will create exact copy of A company with name B.

Of course, test company B will not have a Record Links, as they had same ID, therefore not overwritten, but company still A in Record Link table.

Seems like design flaw in 2009 – but do we care about this? More interesting: it is the same Record Link table structure in latest versions. OK, we can’t copy company in NAV 2013, but can in NAV 2013 R2 and NAV 2015. I’ll investigate and report later how this behave in modern NAV.

Dynamics NAV 2015 and Windows 8.1 (Not PRO)

I’ts turned out that you can’t install Dynamics NAV 2015 web client components on Windows 8.1 not PRO edition. The bloody thing does not  support Windows Active Directory, so can’t enable windows authentication for IIS, which is prerequisite fro WEB client. So, MSFT just made another £100 of me 🙂 Yeah, upgrade to PRO costs just £100.

So, buying a new laptop – check what version of windows is installed on it!

Dynamics NAV and SQL Linked tables

While fiddling with my dozens of linked tables I’ve came across an interesting issue.

I’ve got a table with simple primary key (field 1). 1. This table is connected to SQL View, and the View is pointing to SQL Table. When I run some code against data in my table I’ve identified that there is number of records with same primary key! Well, it is not something you can see in NAV world.

I did some investigation on this, and found, that my SQL table (which wasn’t created by me, but provided by third-party organisation) has no primary key on it, therefore it is fine to have duplicate values in any (or even all fields) for some (or even all) records.

It seems Dynamics NAV Linked tables can’t handle this situation properly. Well, when you define any table in NAV it has to have a primary key, and you will expect it to be unique.  But not for a linked tables. Linked tables inherits the schema defined in the SQL side, so it may have non-unique values in your primary key field.

I’ve tried to reproduce the issue – and have done it quite successfully, please have a look to pictures below:

  1. NAV table structure and Primary Key
    SQL1
  2. SQL table Definition
    SQL2
  3. Data I’ve put to SQL table
    SQL3
  4. Data I can see in NAV system
    SQL4

So, what we can do to resolve this? In ideal world we should always have same primary key in SQL table and NAV linked table. In real life it may be difficult (you may have no right to alter tables provided by third-party organisations) – so if you unable to alter SQL table – then you have to handle this on NAV side by code… Of course, I would prefer to keep same keys on both sides, in Dynamics NAV and in SQL.