Latency is one of the most misunderstood topics in the industry. Ivory itself has very little if nothing to do with latency. The speed of your computer will have a significant impact, but it is mostly introduced by the audio hardware in your system and can be controlled with the hardware buffer size setting in your host software.

In Digital Performer, the setting is called "Buffer Size":
Setup > Configure Audio System > Configure Hardware Driver

In Pro Tools, the setting is called "H/W Buffer Size":
Setups > Playback Engine

In Logic, the setting is called "I/O Buffer Size":
Preferences > Audio > Drivers > Core Audio

Default values are usually set to 256 or 512. Selecting a lower value will improve latency. The trade-off is CPU load, so setting too low may cause other performance problems.

In addition to hardware buffer size, increasing your hardware sample rate will also improve latency.

And finally, some effects plug-ins impose a delay due to the nature of how they process audio. The host may compensate for this during normal session playback, but will be exposed while playing Ivory in real time.