Unbearably Slow

[John Fineberg]

I have 3,600 records in my Tap Forms. Launching the program is unbearably slow. Going from one record to another is unbearably slow. Practically every action is unbearably slow. I’ve done the general maintenance — many times — but vacuuming, rebuilding, etc., doesn’t make a noticeable difference in performance.

We live in an instantaneous world, so sitting and watching a spinning beach ball is like going back to dial-up, when you’re used to high-speed cable. Do you have any suggestions for improving performance?

[Frank Tikket]

I share the same frustration and could not have worded it better myself…. fortunately for us bigger database guys this database does have some incredible features Brendan has built in over the years. We just need to wait (as patiently as possible) for Brendans full database engine overhaul… not sure what the exact timeline is, but with more speed this thing will be A+++++ in every way!

FYI-Ive also noticed speeds are faster on SSD hard drives over spinning HD disks (huge difference on mac) iPhone and iPad seem to still be pretty quick. You can also try to sync your local copy to the cloud, then delete your local copy, then wait for a fresh version to sync back to your mac (do a manual backup first obviously). :-)

[Brendan]

I understand your frustration. The problem may be in the number of fields you have and which view you’re using. If you have say 100 fields in your form and you have 3600 records, that’s 360,000 pieces of information that has to get loaded in to display. Not an easy task.

But I’m working on it. I’m currently investigating migrating over to the Couchbase Mobile database engine. I haven’t gotten far enough yet to do any performance tests with it, but I’m hopeful it will be quicker than what I currently have.

I had started a big database migration project before and the speed improvements were great, but the syncing proved to be nearly impossible, so I decided a little while ago to give Couchbase Mobile a shot to see how it performs. The big bonus with Couchbase Mobile is it has built-in sync ability.

Thanks,

Brendan

[John Fineberg]

Brendan, for a point-of-reference, I have 36 fields and a multi-column view. So nowhere near 100 fields, yet I do have an extensive notes field, in case that is relevant.

[A Nikulin]

I’m with you guys.
Great features except it’s shockingly slow.
Got the latest Mac desktop w/ SSD – barely useable.

[Brendan]

So far the Couchbase migration has proved to perform much better than my previous implementation, so that’s good news for everyone. At least in initial testing.

Thanks,

Brendan

[John Fineberg]

Glad to see you’re making progress, Brendan. Thanks for the update. ~ John

[John Fineberg]

Hi, Brendan, I’m checking in to see what progress you’ve made in the past couple of months in speeding up Tap Forms to what one would expect in this digital world.

By the way, I upgraded one of my computers last night to El Capitan. On that computer, Tap Forms will not load at all. Out of curiosity, I decided to open Bento, to see if it would load in the new operating system. Not only did it load, it loaded extremely fast, as it always did. I only switched away from Bento because FileMaker said it was no longer going to be supporting it. But, even two operating systems later, it’s still outperforming Tap Forms — by far.

I sure hope you’re making serious progress on your updates, as I’m now seriously considering reverting back to a dinosaur that outperforms your current program.

[Brendan]

Hi John,

You’ll need to grant Tap Forms access to the Contacts database in the Security & Privacy settings to get the window to open. Also in 3.5.4 I’ve made some performance improvements when navigating records with the keyboard, especially when viewing a custom layout.

Thanks,

Brendan

[John Fineberg]

Hi, Brendan. I can’t find Security & Privacy -> Contacts database. I’m using 3.5.3. If I need to upgrade to 3.5.4, where are the directions to do that, as I don’t see an “upgrade now” kind of option in your 3.5.3 menu. ~ John

[John Fineberg]

I found 3.5.4 available tonight in the Apple App Store and installed it. I still can’t find Security & Privacy -> Contacts option you referred to anywhere. I’ve tried all of the maintenance tools. Nothing makes my records appear. All I can see is the menu. Nothing shows up beneath it. No window at all. Nothing. And TF keeps crashing, pretty much whenever I try something new to make the window appear. Your thoughts?

[Brendan]

Hi John,

It’s in the System Preferences app, not in Tap Forms. Sorry if I wasn’t clear about that.

Thanks

Brendan

[Frank Tikket]

John,
I had the same size database as you. I too am patiently waiting for the database engine overhaul (Brendan, please do let us know the progress on this-that is what the follow up question was looking for.)

My workaround John, is that I only sync between two devices iOS and Mac (no more iPad)…. Biggest improvement is when I left the HHD and switched to SSD (NIGHT AND DAY) speed difference when clicking smart searches, or jumping forums. I actually have 3900 records with about 100+ fields of various types of data.

Also the reason for two devices, is my database kept growing from the original 200meg to 6 GIG. Yet, when I exported my data it was only a few hundred KB. It’s crazy. I figured when I synced data over 3 devices, the order of the updated data on each device would get lost and tap forms to straighten things out would REWRITE THE WHOLE DATABASE from one device to another from scratch. I assume what that did was great LOTS OF ZEROED out data that made my File size grow (even after all maintenance steps were performed.

The way I run it today (two devices, SSD MAC and 1 iOS device) is still slow compared to modern day speed of things but it works, and it’s usable. As said earlier, I too am interested in the update on the database engine overhaul.

Thanks!

[Brendan]

Frank,

There has been a strange issue arise occasionally (I’ve seen about 3 or 4 cases of this) where the join table I use for managing many-to-many links grows uncontrollably. I can fix it if the database is sent to me. Somehow there are a bunch of rows with incomplete data that get inserted into that join table. Well, a bunch is really like millions. The worst case I’ve seen is an extra 19,000,000 rows got inserted into that file. I was able to reduce the size from over 4 gigabytes to down to only a few hundred megabytes after deleting all that extraneous data.

As for my database overhaul, I’m very active working on it (in between bug fixes to the current version to deal with these El Capitan issues). I probably mentioned this one before, but one test I did was to import a Bento file which had 117,000 records in it. The current version took 1.5 hours to import that much data. The new version took 2 minutes. The amount of disk space used was way less as well.

I’m still working on just the Mac version at the moment. I want everything there to be perfect in regards to the app still working and of course syncing between different Macs. I’ve finally managed to get syncing working, but there are issues with refreshing data that I’m working through.

Once I get that all perfect, then I’ll begin the migration of the iOS versions. I also still have to figure out what cloud solution to provide. I’ve just been working on WiFi sync at the moment because it doesn’t require a cloud service. iCloud and Dropbox can’t be used with the synchronization built-in to Couchbase because there must be a CouchDB based back-end to communicate with. I’ve actually been contemplating a dual service approach. All file attachments and photos could possibly be stored on iCloud or Dropbox, and all database data would be stored with some other online CouchDB based service. IBM has a solution called Cloudant which looks very interesting and is a possible candidate.

What’s really cool though is syncing between my two MacBook Pros is nearly instant. That is, I enter a value on one Mac and a moment later that value shows up on the other Mac. Or pick a photo on one Mac and by the time my head turns to look at why other Mac, the photo is over there already. That’s on WiFi sync of course, so no cloud to cause any delays. But it’s very cool.

So anyway, there’s a status update on what I’m currently working on.

Thanks,

Brendan

[John Fineberg]

If I’m understanding this correctly, it’s impossible to sync through iCloud at the moment. If so, I’ll stop trying. I thought I was doing something wrong, as I’ve tried, unsuccessfully, countless times.

What are the steps to syncing via WiFi?

By the way, I’m also having a problem even reaching this forum. Login attempts keep failing, no matter how many times I try, and no matter how many different browsers I utilize. I finally tried logging into one of my other forums and got in just fine, then navigated to this forum manually.

[Brendan]

Hi John,

It’s not impossible to sync via iCloud. Lots of people are doing it. But if you’re having troubles with it, perhaps try Dropbox to see if that works better for you.

My reply above about WiFi sync was what I’m working on. It’s not in a shipping product at the moment. Sorry.

I’m not sure why you’re having troubles with the forum sometimes. Is it the captcha you’re having troubles with?

Thanks,

Brendan

[John Fineberg]

I’ve tried so many things, my head is spinning!

No troubles with Captcha. I go through all of the normal stuff in logging in, but then I scroll down to the bottom of the forum page, and that asks me to log in … which I just did. I’m not a computer novice. Logging in should be the least of my worries. Anyway, as long as I don’t close this window, or I don’t time out, I’m in … for now.

Back to syncing. I used to be able to see all three of my devices (Mac mini, MacBook Pro and iPhone) listed in the Registered Devices window. Now I can get a maximum of two out of three of those. Let’s say I get my MM registered. Then I get my iPhone registered. But then, when I go to my MBP to “Reset Sync” (since it’s not showing up, then “Enable Sync”), I can then see the MBP listed … but then the other two devices disappear in the process. When I go back to Square One, I can once again get both the MM and the iPhone to see each other and sync, yet the MBP then disappears.

I have all three devices set with the same encryption key. I have tried rebooting all three devices, as I saw you suggest in one of your other forums. I have watched your video from Dec. 2013 (https://www.youtube.com/watch?v=Do_X343ZGOk). I have gone to iCloud Drive and deleted Tap Forms (where all of the folders were, and still are, empty). When I go and try to sync my devices through my MM, those folders reappear in iCloud Drive, but they’re still empty, even though I’ve synced my iPhone through it.

I’m guessing that the answer to my syncing problems is something really simple — but not at all obvious to me.

[Brendan]

Hi John,

Well, maybe you’re not waiting long enough before pressing Reset Sync and enabling sync on the other devices. You see, when you press Reset Sync, Tap Forms deletes some files in your iCloud folder. That deletion then propagates to the other devices. But it might take some time. How long I don’t know. It varies from device to device, account to account, moment to moment. That’s Apple’s issue.

So if you were to Reset sync, enable sync, then go to another device and then reset sync on that device, then enable sync, you’re effectively deleting the registration file that was created from enabling sync on the first device.

So take it a little slower. Let iCloud Drive do its thing. Do one thing at a time. You can press Reset Sync on each device, but don’t enable sync on any device until after you’ve done that on all devices. Then wait a couple of minutes. Then, re-enable sync on each device, one at a time. Waiting for it to appear on all other devices. You should see every enabled device appear on each other enabled device. BUT, if there are lots of files on your device that Tap Forms copies to your iCloud Drive folder, those files may be syncing before the registration file finishes syncing over to the other devices. So you might think, “damn, this registration thing isn’t working”, but in fact it is, it’s just taking longer than you’d like.

So give the devices some time to propagate the registration files from device to device before you press the upload to cloud button on any of them. See how that goes.

Thanks,

Brendan

[John Fineberg]

Here’s the message I keep getting when trying to sync the MacBook Pro:

Synchronization has failed
Please try the following:

1. Make sure you have the latest version of Tap Forms installed on all your devices.
2. Make sure you have the same encryption key set on all your devices.
3. Reset Sync.
4. Re-enable sync on all devices.
5. Try again.

These instructions contradict your instructions to do only one. Don’t they?

I reset the sync and re-enabled the sync hours ago. It’s hard to be any more patient than that. Yet it just keeps spinning and spinning.

[Mark Wilkinson]

Hi, finding Tap Forms to be running very slow on my iMac especially after the recent updates. I get the spinning wheel every time it opens a record, or for any action for that matter. The database only has 87 records at the moment, about 700mb, so it’s hardly too big. It’s almost to the stage of being unusable. Do not have the ‘slowness’ with the same files on may iPad or iPhone. Is their a solution?

[Brendan]

Hi Mark,

Just out of curiosity, do you have any images in the multi-column list view showing?

Does the performance improve if you turn off any photo fields in the multi-column list view?

Thanks,

Brendan

[Todd Wright]

It has been a while since the last response so I am wondering if this continues to be a problem for anyone else. I have extremely slow startup on the program. I can get it running ok once started as long as I leave some of the more complicated layouts aside. Graphics cause a definite slowdown. The other thing I have discovered is that if I exit the program with multiple (4) databases running, the next startup of the program will be slow due to opening the 4 databases. If I close all but one down, it will be a bit faster next time… but still quite slow. I now go and have a coffee (literally brew, pour and drink) before everything is up and running. If I open my 4 main databases the whole system bogs down and all of my open computer applications get slower. Perhaps more RAM would help? Any thoughts/solutions would be welcome.

[Brendan]

Hi Todd,

Definitely RAM and a fast SSD will help the opening times.

But you should also try the Compact Database function to see if that helps quicken the opening of the documents.

To run that, go to the Preferences window, click on Maintenance, then click on the Compact Database button.

That should do it.

Thanks,

Brendan

[obisbis]

Fyi…I noticed TapForms slowed down when having an image pasted inside of a Note field. A work-a-round that helped me with performance was to add a Section Header field above the fields that contained images and collapsing the header while browsing through records.

[Brendan]

Oh ya, images within Note fields are a huge performance drain.

[Sam Moffatt]

I’ve run into issues with images in notes fields causing record size to blow out as well as the Apple formatted text implementation when storing binary images isn’t the most efficient. In one place I ended up having an image field next to the notes field to put any images in. That brought the record size down again and made things quicker.

[Brendan]

Since the data for Note fields are stored directly within the record as JSON encoded values, the rich text notes are first converted to binary data, then converted to strings with base 64 encoding. That makes them really big. Plus then to display them, they have to be decoded.

[Author]

Posts