Updating uploaded NP catalogues without Publishing

<< NP's paths and folders ...
Back to Portfolio FAQ index
NP ports and OS X Server ... >>

A constant source of frustration to non-NP Server users is the fact that as catalogue sizes grow, so does the upload time - especially if uploading originals and previews. During this process the website is unavailable to users and the error message shown in a browser's window is wrong (to anyone except a software engineer!). You see either:

Moreover, if the Publishing session fails, e.g. connection to the NP web server is lost, the process must start again and meaning the website is out - with a meaningless message - until a successful upload is completed. Of course the bigger your catalogue the longer the process and the greater the likely chance of failure is uploading to a server outside your own LAN or VPN. I believe this is one of the reasons the v7 NP Unlimited was withdrawn. The view is that small (NP 5-user) users have small catalogues so the above doesn't need addressing. Of course reality is slightly different!

OK, enough of the issue - must we always use the Publish method to upload? In principle, "Yes", because during the upload Portfolio makes a number of changes to the catalogue:

  1. If Previews are being uploaded, the path to the previews folders is set to the local OS path to the site's /previews/ folder as on the web server.
  2. If Originals are uploaded, the uploaded files, to avoid possible name clashes, append the Record ID and a hyphen to the front of the uploaded original's filename. Thus record #13 "mypic.jpg" is uploaded as "123-mypic.jpg".
  3. If Originals are uploaded, the 'Path' field is reset to point to the local OS path to the site's /originals/ folder as on the web server and using the RID-Filename version of the file's name.

Ack! Well, you can write a script to rename copies of the originals and upload them but the previews are a harder problem. There is no way to script-set the location of the previews folder and the remote location may not easily be replicated locally. But wait, why don't you just download the existing catalogue, drag new items into it, and copy across their previews? Unfortunately, the problem is that Record IDs change as records move between catalogues.

Then the penny dropped. Why not just install NP as a demo locally? It doesn't matter if the demo expires as you'll recall it then functions as a 1 IP-per-hour (i.e. single user) installation. Ideal - you are the only person who needs to look at the site created but you now have a local clone of your site ready to upload. There are some assumptions:

Disclaimer. This is a workaround - so please no griping about the work involved - after all, you did want a way round Publish/Upload. I've used the technique successfully on live commercial sites but it is unsupported - so please don't ask Extensis Tech Support for help, as they won't know or understand (unless they've read this!) and they won't help - or rather aren't obliged to do so. So, if you're nervous trial the method first. It does work, but you're on your own if you break anything - start with copies until confident and you'll be fine. If your target deployment NP server has a password for publishing new sites is set in its NP Administration dialog (i.e. you have to give a password before you can publish a new site to it), make sure to read the section further below.

Given the above assumptions, install NP and add your local machine as a new NP server. Name it 'local' and use the loopback IP ''. Now publish your site to the local machine using the same site name as you use online.

You can do the next few steps with the current site still live.

Ideally, after checking your data, recover the catalogue as this will compact it for faster upload and take less space online. This step is not necessary to achieve the technique below.

Make a note of the new record IDs and upload the new records' previews/originals from your local NP site. Alternatively use the synchronisation features of your FTP client to make sure the remote folder reflects the local one.

If you've altered your templates at all you can also upload these - they'll be live immediately.

Now make a copy of your local site's catalogue. Do a simple rename, e.g. add an 'x' on the front of the filename. This is so that when you upload, as you'll now do, the catalogue doesn't interfere with the current website's catalogue.

Once previews, originals and a copy of the new catalogue are uploaded, it's time to swap over the catalogues. This is where you'll disable the site briefly - to allow the catalogues to be swapped over. The next step depends on whether you have NP admin access via your client. If you do have admin access:

  1. Open the NP admin dialog and switch to the 'Manage Sites' tab.
  2. Un-tick the 'enable' box for your website.
  3. In Windows, click 'Apply'. In Mac, click 'OK' and then re-open the NP Admin dialog to the same panel.
  4. Wait about 30-seconds then check with a browser that the site's unavailable.
  5. In your FTP client, change the name of the current online catalogue and rename the uploaded catalogue to the existing catalogue's original name. See also notes below about an alternate process if RID values have changed [see below re an alternate process here].
  6. Tick the 'enable' box for your website.
  7. In Windows, click 'Apply'. In Mac, click 'OK' and then re-open the NP Admin dialog to the same panel.
  8. Check in your browser that the site functions correctly and the new content shows up.
  9. You can now delete the old catalogue or download it for archival.
  10. Done - and the site was probably offline for less than 5 minutes.

You may have issue with wrong data being shown if the new catalogue has changed Record IDs from the currently served catalogue. In such cases there is an alternate step #5 where we need to set the alias to point to a differently named catalogue. To do this, with the site selected in the admin list:

From experiment, the above process causes NP to flush the old catalogue from memory and load the new one.

If you don't have NP admin access but do have FTP access, there's a simple hack you can use. Download the "site.properties" file from your site and open it in a plain text editor (Notepad, TextEdit, BBEdit but not Word or the like). Some FTP clients, like Transmit, allow you to edit the files directly on the server, saving a download. Look for the line:

Enabled = 1

Instead of step #3, change this line to

Enabled = 0

Save the file and upload it to your site (unless editing directly on the server). Then follow the above steps until #7. You now reverse the process, setting

Enabled = 1

Save and upload and carry on. If you need to reset the catalogue path (step #5 above) then it's actually easier than via the admin dialog. Download and open the "alias.properties" file (which is plain text format). Locate the path string for your catalogue, usually the line starting "catalog.1.path" and simply edit the path string so it uses the new catalogue name. If using the FTP-based upload, I strongly advise changing the FDB name each time it is uploaded. It both avoids naming collision and ensures the old catalogue data is flushed from memory.

If you work in a corporate environment or have availability commitments you might want to err on the side of caution during the above processes and advertise your site will be unavailable for 15-30 minutes (whatever duration you feel comfortable with). However, once you get the hang of the technique you can pre-deploy all your assets with the offline switchover taking a few minutes at most.

One other benefit of this technique for swapping out catalogues is that you can now test and tweak your site without needing lengthy uploads. If space is at a premium locally, you can delete the local site once you're happy the upload is complete.

Specimen paths

Assuming the site name is 'mytest' and a sample image "mypic.jpg" with RID #123, here are the likely Mac preview & original paths. Note the odd mix of folder delimiter styles on Windows:

What if I work cross-platform?

Well you're out of luck. Actually, there are some solutions! The simplest - if you're prepared to not use uploaded previews and let NP create them dynamically is to simply script the 'Path' field data, as that is possible via a single OS scripting interface. You are advised to do a test publish of a small catalogue and download it to see the values you need to set.

If you must have previews, you'll need to create a custom field in your local and uploaded files and write a script that copies the Record ID to the custom field. Now, when you copy the records across you know the old RID. You can now write a script to copy/rename existing previews to the RIDs needed online. Use the above method to script the original 'Path' data and set-up a local, correctly renamed /originals/. Then use the technique above.

Using this technique if the NP server has a password for publishing new sites

The NP Administration dialog allows the admin to set 2 passwords. The one you'll know best is the password to access the Administration dialog. The second, when set, requires a password the first time (only) that a new site is published to that NP server. The process results in the 'site.sentinel' file you may see in the root of your NP sites. Note that this file/password submission can't be by-passed via FTP upload - it's no good mimicking the process and just uploading the resulting 'site/sentinel' file. Presumably this is because creation of the site is monitored/stored somewhere in the NP app (collection db?). Anyway, it does mean you need to do your first upload somewhat differently:

Closing thought What were they thinking at the design phase of NP? After all, as the /originals/ and /previews/ are fixed locations inside the uploaded site, why hard-wire changed paths inside the uploaded catalogue? A classic over-engineered solution. If the preview location could be changed by AS/VB scripting then much of this aggravation could be worked around. Even better would be if NP just took the RID and/or filename and constructed paths on the fly - after all it knows exactly where the assets are.

Keywords (to assist indexing/searching):

Question: Updating uploaded NP catalogues without Publishing [FAQ00378.htm]
Last Update:- 25 June 2009

<< NP's paths and folders ...
Back to Portfolio FAQ index
NP ports and OS X Server ... >>

Quick Search of PortfolioFAQ (word, words or phase): or try the Advanced Search

User-to-User Forums  |  Report error/typo/broken link  |  Request new topic  |  Ask a question

Site and articles © Mark Anderson 2001-2007 - Visit my home page

This FAQ is created and maintained using
Friday logo
Friday - The Automatic FAQ Maker