Notes for how to use Plone, Zope, HTML, and LAR server

While there are many Plone and Zope manuals, it is nice to have a place for notes for those maintaining the site. Feel free to add your notes below. This page also includes notes on HTML and running the Los Altos Robotics server.

Michael's Notes

Plone

  • The Plone.org documentation site is a great place to start looking for answers. It has a What documentation should I read? page.
  • Manage the Plone site from http://losaltosrobotics.org:8080/Main/plone_control_panel.  This is the same interface as http://www.losaltosrobotics.org:8080/Main/@@overview-controlpanel.
  • I packed the data base on Aug 21, 2013 from http://www.losaltosrobotics.org:8080/Main/@@maintenance-controlpanel
  • There are log files in /usr/local/Plone-4.3.1r1/zinstance/var/log
  • The Navigation object in the left frame is configured by clicking on the Manage Portlets link at the bottom of the left frame. The contents for the left and right frames are controlled from here and what is them is customized from here too. For navigation, at the /Main level, I selected "Include Top Node", "0" for Start level, and "3" for Navigation tree depth.
  • I went to http://losaltosrobotics.org/@@manage-portlets and hid the calendar, recent changes, and news portlets from the left frame.  Michael 1/31/15.
  • The FLL folder was not showing up in the left frame.  A listing of the contents showed that the FLL page and other pages were expired.  I selected the expired pages and clicked on "Change State" near the bottom of the frame and saved the different pages as "published".  For some of them, this was enough.  For others, I had to set the date to something like Jan 1, 2020 12:00 pm before it would work.  I went back to those and saved them with a comment and no Expiration date to try and clear the expiration date.  Hopefully this works.  The pages still showed.  I did not see a way to show the expired date.  I did not look to hard either though.  Michael 1/31/15.
  • To remove the "leave comments" feature at the bottom of the main part of a page, go to the "edit" tab and then select "settings" from the row of options that show up at the top of the edit page, and then deselect "Allow comments"
  • You can manage ZOPE from http://www.losaltosrobotics.org:8080/Main/manage and http://www.losaltosrobotics.org:8080/Main/manage_main
  • Working Copy lets you have two versions of your content in parallel.
  • Unlocking locked Plone objects and Locked content. I also learned that if you add "/edit" after the URL, you should get a screen you can edit the file from, change ownership, and more. I deleted a file that was locked and owned by eitan by changing the owner to me and then deleting it.
  • The Data.fs file contains everything that is in the plone website. It is a database file so it needs to be backed up when it is not being changed.
    • Backup and recover Data.fs in linux. Here is the Zope way of backing up the Data.fs site database.
    • Dave told me about some tools for working with the Data.fs file. Documentation on fstest (and others). I tried running analyze.py in /usr/lib/zope2.10/lib/python/ZODB/scripts but had to do it by first setting PYTHONPATH.
      export PYTHONPATH=/usr/lib/zope2.10/lib/python

      python2.4 ZODB/scripts/analyze.py /var/lib/zope2.10/instance/plone-site/var/Data2.fs
    • Discard old version of objects in the ZODB. Using this technique will remove all of the old version of objects that are needed to allow you to "undo" changes. I don't know if it has ever been done. To do it you would connect to http://schuh.homelinux.com:8081/manage . You probably have to use one of the administrators user names and password. pick "Control_Panel" in the left pane, then pick "Database Management" from the list at the left of the right hand pane.
    • I moved the Los Altos Robotics website in August 2010 by copying the Data.fs file to a new server. I pointed plone at it and it came up fine. I read Moving a Plone siteto come up with this approach.
      • apt-get install plone3-site (this installs a working plone site)
      • unpacked a copy of the plone instance from svn.losaltosrobotics.org
      • Linked /var/lib/zope2.10/instance/plone-site/var to the LAR version. var -> /var/zope/lar.env/lar.buildout/var
      • restarted zope: /etc/init.d/./zope2.10 restart
      • waited
  • Hardening Plone
  • YouTube Videos
    • Insert the videos using the little "filmstrip" "insert/edit media" icon and selecting "media_dlg.iframe" video Type.  Turn on the "insert/edit media" icon by going to the TinyMCE editor http://www.losaltosrobotics.org:8080/Main/portal_tinymce/@@tinymce-controlpanel control panel, selecting the toolbar tab, and enabling it.  There are lots of other good options on the TinyMCE editor control panel.  Setting the toolbar width to 800 pixels makes it take up less vertical space when editing.  Michael 9/1/13.
    • Changing the video links to have "embed" in the path like "http://www.youtube.com/embed/u7k5IxsixO4" allows them to be shown full screen.  I bet this will work with most YouTube videos.  I could not figure out how to get "allowfullscreen" to work as a modifier.  It kept getting removed.  Michael 9/1/13.
    • YouTube videos quit working after the 4.3.1r1 upgrade.  I read through the How to Embed Flickr, YouTube, or MySpace Content page to understand how to enable the YouTube videos to be shown.  I found that if the videos were inserted as an iframe, they worked.  I decided to see if we could avoid relaxing the security settings and still show the videos by using iframe.  This worked.  Michael 9/1/13.
    • I read the directions in How to Embed Flickr, YouTube, or MySpace Content to configure the site to show the videos that Edmond links in. While I made the correct changes in the HTML Filtering configuration, it did not take effect right away. It did not work for maybe an hour after I did it but did work the next morning. I had other changes work quicker but not right away.  (This note was here long before 9/1/13. I found it after figuring out the iframe fix.)
  • Customizing Plone 3: You can customize Plone by changing the settings in the ZMI (through the web) or by making changes to an installable filesystem product.
  • Versioning is available by default in Plone. The history tab did not show on the pages when I first started working with the version that Jonathan ported to Plone 3.0 for us. I was able to add the history back in to the schuh.homelinux.com:8081/Main version by going to http://schuh.homelinux.com:8081/Main/manage and copying and pasting one of the existing object types like "Contents" in Main/portal_actions/object (left frame) and updating it to have
    • Title: History
    • I18n Domain: plone
    • URL (Expression): string:${object_url}/versions_history_form
    • Icon (Expression):
    • Condition (Expression): python:portal.portal_repository.isVersionable(object)
    • Permissions: Modify portal content
    • Visible?: checked
    Now the history and versioning works. I want to see if I can make it use the nicer visual editor. I think I will be done after that.
  • There is a tool withing plone for packing. It is also possible to pack Data.fs offline if needed.
  • Dave helped me understand page permissions after I asked why eitan owned the homepage. He wrote "I can say something about the eitan part, the objects are owned by who every created them. And that doesn't change when you edit. Maybe there is a way to configure it to show the last editor. I couldn't figure that out in the old Plone version. There might have been a change in the new version since it seems obvious that the last editor is the right thing.
    In the old version you had create a copy and then replace the original to change the creator's name. Or you can take ownership of an object as an administrator. but that seems like too many steps."
  • Members/User
    • Login at http://www.losaltosrobotics.org:8080/Main/login_form
    • Logout at http://www.losaltosrobotics.org:8080/Main/logout
    • See the ZOPE section below for more on users.
    • Change the logged in users password at http://www.losaltosrobotics.org:8080/Main/password_form. Note: we don't have email setup on the server, so password reminders can not be sent. If you have lost your password you will need to have an administrator reset it.
    • http://www.losaltosrobotics.org:8080/Main/Members/folder_contents will show you a list of accounts.
    • Going to site setup and selecting Users and Groups will take you to http://www.losaltosrobotics.org:8080/Main/prefs_users_overview which will also show you all of the users.
    • http://www.losaltosrobotics.org:8080/Main/Members/michael shows michael's files.
    • Steps for creating a new LosAltosRobotics.org website user:
      • Go to /Main/acl_users/source_users/manage_users and click on "Add a User".  Add the user from the page that comes up.
      • Go to http://www.losaltosrobotics.org:8080/Main/prefs_users_overview and click on show all.
        • Click on the users name and update the name and email address.
        • Go back to the prefs_users_overview screen and check "Contributor, Editor, Member, reader, and reviewer" boxes and then click on apply changes.
  • Plone 4 update for Plone Hotfix 20110928 - October 4, 2011
    • The directions were hard to follow.  I tried the recommended way but could not get the server to run in the foreground and show me that the hotfix was installed.  I found Jonathan's comments from July 23, 2011 email.  I could not get this to work.
      • Ok.  Fun times!  Turns out that the Plone documentation massively oversimplifies what the unified installer does. 
        If you install Plone with a standalone Zope+ZODB instance, the documentation is correct and the instance management script is:
        bin/instance
        If you install Plone with ZEO, the documentation is wrong and the instance management script is:
        bin/zeoserver
        It looks like bin/zeoserver takes the same arguments as bin/instance.
    • I decided to try the buildout path to installing the patch and was able to get that to work.  It was actually pretty simple.  Hear are a few lines from the buildout.cfg file that explain what I did:
          71  ############################################
          72  # Eggs
          73  # ----
          74  # Add an indented line to the eggs section for any Python
          75  # eggs or packages you wish to include.
          76  #
          77  eggs =
          78      Plone
          79      Pillow
          80      Products.Zope_Hotfix_20110622
          81      Products.PloneHotfix20110928
          82
          83  # I installed the 20110928 hotfix by following the directions at
          84  # From http://plone.org/products/plone-hotfix/releases/20110928
          85  # I did not download the zip file anywhere or expand it.  The
          86  # "./bin/buildout -Nv" command downloaded it and installed it
          87  # after I added the "Products.PloneHotfix20110928" line to the
          88  # "buildout.cfg" file.
          89  # Michael Schuh, Oct 4, 2011.
    • Here a few lines from the "./bin/buildout -Nv" command that show that the "egg" was downloaded and installed.  Magical in my opinion.
      Updating zeoserver.
      Installing client1.
      Installing 'Plone', 'Pillow', 'Products.Zope_Hotfix_20110622', 'Products.PloneHotfix20110928', 'plone.recipe.zope2instance'.
      We have the distribution that satisfies 'Plone==4.0.7'.
      We have the distribution that satisfies 'Pillow==1.7.2'.
      We have the distribution that satisfies 'Products.Zope-Hotfix-20110622==1.0'.
      We have no distributions for Products.PloneHotfix20110928 that satisfies 'Products.PloneHotfix20110928'.
      Getting distribution for 'Products.PloneHotfix20110928'.
      Running easy_install:
      /usr/local/Plone/Python-2.6/bin/python "-c" "from setuptools.command.easy_install import main; main()" "-mUNxd" "/usr/local/Plone/buildout-cache/eggs/tmpAgPHkf" "-Z" "-q" "/usr/local/Plone/buildout-cache/downloads/dist/Products.PloneHotfix20110928-1.1.zip"
      path=/usr/local/Plone/Python-2.6/lib/python2.6/site-packages/distribute-0.6.16-py2.6.egg

      Got Products.PloneHotfix20110928 1.1.
      Picked: Products.PloneHotfix20110928 = 1.1

      We have the distribution that satisfies 'plone.recipe.zope2instance==4.1.5'.
      Getting required 'Zope2==2.12.18'

  • Plone 4.0.7 installation - July 2011
    • Version Numbers:
          Plone 4.0.7
          Zope 2.12.18
          Python 2.6.7 (r267:88850, Jul 23 2011, 20:41:06) [GCC 4.4.5]

    • Download the latest Plone release
      • http://launchpad.net/plone/4.0/4.0.7/+download/Plone-4.0.7-UnifiedInstaller-Hotfix-20110622.tgz
      • drwxr-xr-x 10 root       root           4096 Jun 30 12:10 Plone-4.0.7-UnifiedInstaller
      • -rw-r--r--  1 root       root       39642880 Jun 30 12:23 Plone-4.0.7-UnifiedInstaller-Hotfix-20110622.tgz
      • cd Plone-4.0.7-UnifiedInstaller
    • apt-get install gcc g++ make libssl-dev
    • unpack it and then run: ./install.sh zeo
    • sudo -u plone /usr/local/Plone/zeocluster/bin/plonectl start
    • sudo -u plone /usr/local/Plone/zeocluster/bin/plonectl stop
    • Change the port (typically 8080) by editing /usr/local/Plone/zeocluster/buildout.cfg.   After updating, be sure and stop plone, rebuild, and start plone.
      • sudo -u plone /usr/local/Plone/zeocluster/bin/plonectl stop
      • /usr/local/Plone/zeocluster/bin/buildout   # Use to make port changes in buildout.cfg take effect. Stop and then start server aferwards.
      • sudo -u plone /usr/local/Plone/zeocluster/bin/plonectl start
    • Backups: I have configured some cron jobs to back up the plone data base at 1350 and 2350 every day.  I configured my machine at home to back up the backup each day at 1359 and 2359 so we will be covered if the machine dies.  I still need to configure the server to start back up after a power cycle.  I will work on that after I return home and when Eitan is home in case I need to go poke it.
    • I used the control panel to turn off filtering of "font" and "embed" tags and enabled "font", "embed", and "iframe" (used for the Google Calendar on the Los Altos Board of Directors Page) as a special tags.  http://losaltosrobotics.org:8080/Main/@@filter-controlpanel .  I had both websites up while sorting this out. http://losaltosrobotics.org:8090/Main/@@filter-controlpanel .  It took several hours for these changes to take effect.  The font tag is what we use for color in the website body.
    • New site: http://losaltosrobotics.org:8080/Main/ and old site: http://losaltosrobotics.org:8090/Main/ 
    • Jonathan's notes:
      • Adding the world icons back in is going to be a bit trickier since the external link styling classes that Plone used to have for displaying those have been removed in Plone 4.  Re-adding them will require adding a bit of JS to the site or adding some sort of server-side custom content filter to Plone.

        But since I promised you a write-up tonight, here's what I've done:
        • To get the "Navigation" title on top of the navigation portlet in Plone, I went over to http://losaltosrobotics.org:8080/Main/index_html/@@manage-portlets (there's a "manage portlets" button at the bottom of the left column, clicked on the navigation portlet, and then manually added the title of the portlet back in.
        • To get the styling tweaks in place for the background color and the bullet color, I had to go into the ZMI (http://losaltosrobotics.org:8080/Main/portal_skins/custom/manage_main) and make changes to my existing theme tweaks; in Plone 4, some of the markup was changed, which broke some of the CSS tweaks.
          • I changed the classes for the initial CSS style in ploneCustom.css to match the new Plone 4 equivalents of what was in there originally.
          • I took bullet.gif, downloaded it, converted it to a png, and created a new Image object in that custom folder entitled "bullet.png".  Apparently, in Plone 4, they switched all of the icon files to png, presumably to reduce load time (since, in most cases, png is smaller).
        • Sorry for taking forever to get around to this.  I got the styling of the navigation sidebar item to be pretty close (not *quite* pixel-perfect, but as close as I could easily get it to be) to the one on the older Plone 3 site.

          I did this by adding some style tweaks to /Main/portal_skins/custom/
          ploneCustom.css via the ZMI.  I also added some additional comments about what each of the tweaks in that document are for.
  • Plone 4.2.0.1 Installation - July 15, 2012
    • Version Overview
         Plone 4.2.0.1
         Zope 2.13.15
         Python 2.7.3 (default, Jul 14 2012, 15:20:35) [GCC 4.4.5]

    • Followed the instructions above until I forgot about the blobstorage.  Then Jonathan again helped me.  Here is what I did.
    • Download the latest Plone release
    • wget https://launchpad.net/plone/4.2/4.2/+download/Plone-4.2-UnifiedInstaller-update-1.tgz
    • tar -xvf Plone-4.2-UnifiedInstaller-update-1.tgz
    • mv Plone-4.2-UnifiedInstaller Plone
    • cd Plone
    • ./install.sh zeo
    • Copy the directories filestorage and blobstorage from the currently active server to /usr/local/Plone/zeocluster/var .  The blobstorage is very important as it has the images in it.
    • Start it with: sudo -u plone /usr/local/Plone/zeocluster/bin/plonectl start
    • If needed, stop it with: sudo -u plone /usr/local/Plone/zeocluster/bin/plonectl stop
    • While running, login.  The site should show the pages.  It still needs to be upgraded from the prior version to the current version.
    • Update the database by logging into the management interface.  At the top of the Main page, there should be a message like "The site configuration is outdated and needs to be upgraded. Please continue with the upgrade."  Click on "Please continue with the upgrade" and read the output to make sure everything went ok.
    • If needed, change the port (typically 8080) by editing /usr/local/Plone/zeocluster/buildout.cfg.   After updating, be sure and stop plone, rebuild, and start plone.
      • sudo -u plone /usr/local/Plone/zeocluster/bin/plonectl stop
      • /usr/local/Plone/zeocluster/bin/buildout   # Use to make port changes in buildout.cfg take effect. Stop and then start server afterward.
      • sudo -u plone /usr/local/Plone/zeocluster/bin/plonectl start
  • Plone 4.2.0.1 Installation - August 20, 2012.  This time with zinstance (single cpu) rather than zeo (cluster configuration)
    • Version Overview
         Plone 4.2.0.1
         Zope 2.13.15
         Python 2.7.3 (default, Jul 14 2012, 15:20:35) [GCC 4.4.5]

    • Followed the instructions above until I forgot about the blobstorage.  Then Jonathan again helped me.  Here is what I did.
    • Download the latest Plone release
    • wget https://launchpad.net/plone/4.2/4.2/+download/Plone-4.2-UnifiedInstaller-update-1.tgz
    • tar -xvf Plone-4.2-UnifiedInstaller-update-1.tgz
    • mv Plone-4.2-UnifiedInstaller Plone
    • cd Plone
    • time ./install.sh standalone   # This took 1.5 hours to run on losaltosrobotics.org (lar)
    • cd /usr/local/Plone/zinstance to add the plone.app.caching egg.
      • I added plone.app.caching to /usr/local/Plone/zeocluste/buildout.cfg to the zcml section and uncommented http://dist.plone.org/release/4.2-latest/versions.cfg (the first time I did this, I uncommented out the http://dist.../versions.cfg line and the last time I did not.  It worked fine without it being uncommented).  I tried adding http://good-py.appspot.com/release/plone.app.caching/1.0b2 but this failed due to a version conflict when running ./bin/buildout so I took it out and tried again.
        • diff -c -r1.1 ./buildout.cfg
          --- 108,114 ----
            # Some eggs need ZCML slugs to tell Zope to
            # use them. This is increasingly rare.
            zcml =
          +     plone.app.caching
            #    plone.reload
      • time ./bin/buildout  # This took 4 minutes
      • /etc/init.d/plone4_server_mschuh stop
      • ssh losaltosrobotics.org; cd /usr/local/Plone/zeocluster/var; tar -czf /tmp/plone.tgz filestorage.LosAltosRobotics blobstorage
      • cd /usr/local/Plone/zeocluster/var
      • mv filestorage.LosAltosRobotics filestorage.LosAltosRobotics.FirstTry
      • mv blobstorage blobstorage.FirstTry
      • mkdir secondTry
      • scp michael@losaltosrobotics.org:/tmp/plone.tgz secondTry
      • tar -tzvf secondTry/plone.tgz
      • /etc/init.d/plone4_server_mschuh start
      • browse to http://192.168.0.70:8080/Main/@@overview-controlpanel and click on "Add-ons" in the "Plone Configuration" section.  Select "HTTP caching support 1.1/Installs plone.app.caching" and then click on "activate" at the bottom of the list.
      • browse to http://192.168.0.70:8080/manage and upgrade the site.
    • Copy the directories filestorage and blobstorage from the currently active server to /usr/local/Plone/zeocluster/var .  The blobstorage is very important as it has the images in it.
    • Start it with: sudo -u plone /usr/local/Plone/zeocluster/bin/plonectl start
    • If needed, stop it with: sudo -u plone /usr/local/Plone/zeocluster/bin/plonectl stop
    • While running, login.  The site should show the pages.  It still needs to be upgraded from the prior version to the current version.
    • Update the database by logging into the management interface.  At the top of the Main page, there should be a message like "The site configuration is outdated and needs to be upgraded. Please continue with the upgrade."  Click on "Please continue with the upgrade" and read the output to make sure everything went ok.
    • If needed, change the port (typically 8080) by editing /usr/local/Plone/zeocluster/buildout.cfg.   After updating, be sure and stop plone, rebuild, and start plone.
      • sudo -u plone /usr/local/Plone/zeocluster/bin/plonectl stop
      • /usr/local/Plone/zeocluster/bin/buildout   # Use to make port changes in buildout.cfg take effect. Stop and then start server afterward.
      • sudo -u plone /usr/local/Plone/zeocluster/bin/plonectl start
  • Plone Hotfix 20121106 (Nov 06, 2012) - followed buildout directions
    • cd /usr/local/Plone/zinstance
    • /etc/init.d/plone4_server_mschuh stop
    • vi buildout.cfg ; add
      rcsdiff -c ./buildout.cfg
      ===================================================================
      RCS file: ./RCS/buildout.cfg,v
      retrieving revision 1.2
      diff -c -r1.2 ./buildout.cfg
      *** ./buildout.cfg      2012/11/06 15:58:54     1.2
      --- ./buildout.cfg      2012/11/06 16:01:00
      ***************
      *** 76,81 ****
      --- 76,82 ----
            Plone
            Pillow
            lxml
      +     Products.PloneHotfix2012110
    • To install the 1.2 update, I had to move the 1.0 version out of the way.
      • mkdir save
      •  mv ../buildout-cache/eggs/Products.PloneHotfix20121106-1.0-py2.7.egg ../buildout-cache/downloads/dist/Products.PloneHotfix20121106-1.0.zip save/
      • (time ./bin/buildout -Nv ) > buildout_20121107.2ndTry.out &
    • /etc/init.d/plone4_server_mschuh  start
  • Upgrade to Plone 4.3.1r1 from 4.2.0.1 Installation - July 28, 2013
    • Notes on Plone 4.3.1r1 upgrade. This is the first version where we embraced the TinyMCE visual editor and removed the Kupu Visual editor.

      apt-get update
      apt-get install libreadline5-dev readline-common libxslt1-dev libxml2-dev libjpeg-dev
      apt-get upgrade
      apt-get install poppler-utils wv   #  Installed to try and get rid of update error about pdf_to_text and wv to deal with indexing Microsoft Office Files
      flash-kernel # This loads any new kernel formed onto the boot loader.

      wget https://launchpad.net/plone/4.3/4.3.1/+download/Plone-4.3.1r1-UnifiedInstaller.tgz
      tar -xf Plone-4.3.1r1-UnifiedInstaller.tgz
      cd Plone-4.3.1r1-UnifiedInstaller/
      time ./install.sh --build-python --target=/usr/local/Plone-4.3.1r1 standalone
      real 62m8.698s
      user 48m12.260s
      sys 2m13.840s

      vi buildout.cfg
      Set the port to 8090
      sudo -u plone_buildout bin/buildout
      Ignore the two test error messages. https://dev.plone.org/ticket/13488 says they are unimportant.
      more /usr/local/Plone-4.3.1r1/zinstance/README.html
      to get the startup command.
      sudo -u plone_daemon /usr/local/Plone-4.3.1r1/zinstance/bin/plonectl start
      # Use port forwarding to view the 8090 website.
      Michael[501] CAD: ssh -L 8095:localhost:8090 lar
      # View on CAD by going to http://localhost:8095
      # Probably could have used 8090 rather than 8095 in both commands.
      # This shows that the server works.
      sudo -u plone_daemon /usr/local/Plone-4.3.1r1/zinstance/bin/plonectl stop

      $HOME/bin/backupPloneBlobstorageFiles.rb -0 >> /var/schuh/log/backup_LosAltosRoboticsWebsitePloneDatabasePlone4_blobstorage.log 2>&1
      # Make a full backup of the blob storage.

      tail -100 /var/schuh/log/backup_LosAltosRoboticsWebsitePloneDatabasePlone4_blobstorage.log
      l /backups/linux/lar/plone/blobstorage/blobstorage_lar.*.l0.tgz
      pd /backups/linux/lar/plone # this is where the backups live.
      pd /var/schuh/log
      more backup_LosAltosRoboticsWebsitePloneDatabasePlone4.log
      tail -200 backup_LosAltosRoboticsWebsitePloneDatabasePlone4.log

      mkdir /backups/linux/lar/plone/Plone4.2.0.1.Data.fs # I figured out that I had removed the database file so I had to make a new directory for the backup.
      l /usr/local/Plone/zinstance/var/filestorage/Data.fs # This is where the database file lives.
      crontab -l | more # Get the database backup command and run it.
      $HOME/bin/backup_LosAltosRoboticsWebsitePloneDatabase.plone4 >> /var/schuh/log/backup_LosAltosRoboticsWebsitePloneDatabasePlone4.log 2>&1
      tail -200 backup_LosAltosRoboticsWebsitePloneDatabasePlone4.log

      # Set up the blobstoreage and database directories.
      cd /usr/local/Plone-4.3.1r1/zinstance/var
      mv filestorage filestorage.AsCreatedByInstallScript
      mv blobstorage blobstorage.AsCreatedByInstallScript

      tar -xzvf /backups/linux/lar/plone/blobstorage/blobstorage_lar.130726.2116.l0.tgz
      mv blobstorage.LosAltosRobotics.Plone4.2.0.1 blobstorage.LosAltosRobotics.Plone4.3.1r1
      rm blobstorage
      ln -s blobstorage.LosAltosRobotics.Plone4.3.1r1 blobstorage

      # restore filestoreage from a backup.
      mkdir filestorage.LosAltosRobotics.Plone4.3.1r1
      ln -s filestorage.LosAltosRobotics.Plone4.3.1r1 filestorage
      pd filestorage
      /usr/local/Plone-4.3.1r1/zinstance/bin/repozo -Rv -r /backups/linux/lar/plone/Plone4.2.0.1.Data.fs -o /usr/local//Plone-4.3.1r1/zinstance/var/filestorage/Data.fs.restored3
      diff Data.fs.restored3.index /backups/linux/lar/plone/Plone4.2.0.1.Data.fs/2013-07-27-04-29-37.index
      # they are the same. A copy should work just fine.
      mv Data.fs.restored3 Data.fs
      mv Data.fs.restored3.index Data.fs.index

      # set the file permissions
      chown -R plone_buildout.plone_group /usr/local/Plone-4.3.1r1/zinstance/var/*
      # then run buildout from the /usr/local/Plone-4.3.1r1/zinstance directory.
      cd /usr/local/Plone-4.3.1r1/zinstance
      sudo -u plone_buildout bin/buildout # This does a bunch of chmods to make sure the permission are correct. 

      # Start the server
      sudo -u plone_daemon /usr/local/Plone-4.3.1r1/zinstance/bin/plonectl start

      # Browse to the top level and upgrade the Plone instance(s)
      http://localhost:8095/
      # Then check the log files in var/log to see if there are any errors you want to deal with.


      # Stop the old server


      # Other notes:
      #
      # Manage the site from http://localhost:8095/Main/plone_control_panel
      # Found that the Kupu entry in the http://localhost:8095/Main/plone_control_panel page was mangled so
      # I went back to 4.2.0.1 and went to the Main/plone_control_panel and clicked on the Add_ons link
      # and removed Kupu. There was a message on the 4.2.0.1 addon page that said
      #
      # http://www.losaltosrobotics.org:8080/Main/prefs_install_products_form
      # Kupu 1.5.0
      # Kupu is a cross-browser visual editor.
      # This add-on has been upgraded. Old profile version was unknown. New
      # profile version is 2. There is no upgrade procedure defined for this
      # add-on. Please consult the add-on documentation for upgrade information,
      # or contact the add-on author.
      # Perhaps this is why I get the "Step kupu-setup has an invalid import handler" message
      # in the output report from upgrading the site from 4.2.0.1 to 4.3.1r1.
      #
      # If the permissions are wrong in zinstance/var/{filestorage,blobstorage}, then plone will
      # not start properly. Fix this by
      chown -R plone_buildout.plone_group /usr/local/Plone-4.3.1r1/zinstance/var/{filestorage,blobstorage}
      # Then run buildout from the /usr/local/Plone-4.3.1r1/zinstance directory.
      cd /usr/local/Plone-4.3.1r1/zinstance
      sudo -u plone_buildout bin/buildout # This does a bunch of chmods to make sure the permission are correct.

      # Check the server status by running
      sudo -u plone_daemon /usr/local/Plone-4.3.1r1/zinstance/bin/plonectl status 

    • Aug 21, 2013: Packed the data base from http://www.losaltosrobotics.org:8080/Main/@@maintenance-controlpanel .  Reduced the size from something like 230 MB to 30 MB.
  • Upgrade to Plone 4.3.3 from 4.3.1r1 Installation on robotics.mvla.net - September 27, 2014
    • cd /usr/local
      wget https://launchpad.net/plone/4.3/4.3.3/+download/Plone-4.3.3-UnifiedInstaller.tgz
      tar -tzvf Plone-4.3.3-UnifiedInstaller.tgz
      tar -xzvf Plone-4.3.3-UnifiedInstaller.tgz
      cd Plone-4.3.3-UnifiedInstaller/

      apt-get update
      apt-get install libreadline6-dev readline-common libxslt1-dev libxml2-dev libjpeg-dev
      apt-get install zlib1g-dev
      apt-get install libssl-dev 

      root@robotics:/usr/local/Plone-4.3.3-UnifiedInstaller# time ./install.sh --target=/usr/local/Plone-4.3.3 standalone
      (lots of output)

      real 19m23.592s
      user 16m11.365s
      sys 0m59.496s

      # Restore the plone blob and database.

      cd /usr/local/Plone-4.3.3/zinstance/var
      mv filestorage filestorage.AsCreatedByInstallScript
      mv blobstorage blobstorage.AsCreatedByInstallScript

      # Make a sample listing of the blobstorage backup file and then restore it.
      tar -tzvf /tmp/blobstorage_lar.140926.2108.l0.tgz
      tar -xzvf /tmp/blobstorage_lar.140926.2108.l0.tgz
      tar -xzvf /tmp/blobstorage_lar.140926.2108_140927.0202.l1.tgz

      mv blobstorage.LosAltosRobotics.Plone4.3.1r1 blobstorage.LosAltosRobotics.Plone4.3.3
      mkdir filestorage.LosAltosRobotics.Plone4.3.3
      rm blobstorage
      ln -s blobstorage.LosAltosRobotics.Plone4.3.3 blobstorage
      ln -s filestorage.LosAltosRobotics.Plone4.3.3 filestorage

      cd filestorage
      /usr/local/Plone-4.3.3/zinstance/bin/repozo -Rv -r /tmp/ -o Data.fs

       

      # Make a Data.fs file from the backups.
      #
      # Make sure all of the Data.fs backup files have the same extension.
      # The index file will not be correctly made if all of the files don't have
      # the same date and timestamp.

      mv /tmp/2014-09-27-04-25-43.index /tmp/2014-09-27-04-25-36.index
      -rw-r--r-- 1 michael michael 114 Sep 26 21:27 /tmp/2014-09-27-04-25-36.dat
      -rw-r--r-- 1 michael michael 23661867 Sep 26 21:27 /tmp/2014-09-27-04-25-36.fsz
      -rw-r--r-- 1 michael michael 636494 Sep 26 21:25 /tmp/2014-09-27-04-25-36.index

      cd /usr/local/Plone-4.3.3/zinstance/var/filestorage
      rm Data.fs
      /usr/local/Plone-4.3.3/zinstance/bin/repozo -Rv -r /tmp/ -o Data.fs


      # Set the file ownership to plone_buildout.plone_group
      cd /usr/local/Plone-4.3.3/zinstance/var
      chown -R plone_buildout.plone_group * 

      # Use the buildout script to make sure the permissions are correct
      cd /usr/local/Plone-4.3.3/zinstance/var
      sudo -u plone_buildout bin/buildout
      # This runs the following chmods which sets the permissions so that the plone_daemon user can edit and change files owned by plone_buildout. 

          # Make sure anything we've created in var is r/w by our group
          find /usr/local/Plone-4.3.1r1/zinstance/var -type d -exec chmod 770 {} \; 2> /dev/null
          find /usr/local/Plone-4.3.1r1/zinstance/var -type f -exec chmod 660 {} \; 2> /dev/null
          find /usr/local/Plone-4.3.1r1/zinstance/var -type d -exec chmod 770 {} \; 2> /dev/null
          find /usr/local/Plone-4.3.1r1/zinstance/var -type f -exec chmod 660 {} \; 2> /dev/null
          chmod 754 /usr/local/Plone-4.3.1r1/zinstance/bin/*

      # Start plone
      sudo -u plone_daemon /usr/local/Plone-4.3.3/zinstance/bin/plonectl start

      # Stop plone
      sudo -u plone_daemon /usr/local/Plone-4.3.3/zinstance/bin/plonectl stop

  • Plone Hotfix 20151006 (October 06, 2015) - followed buildout directions
    • cd /usr/local/Plone/zinstance
    • /etc/init.d/plone4_server_mschuh stop
    • root@robotics:/usr/local/Plone/zinstance# rcsdiff ./buildout.cfg
      ===================================================================
      RCS file: ./RCS/buildout.cfg,v
      retrieving revision 1.1
      diff -r1.1 ./buildout.cfg
      73a74
      > plone4.csrffixes==1.0.2
      186a188,190
      > plone.protect = 3.0.12
      > plone.keyring = 3.0.1
      > plone.locking = 2.0.8

    • sudo -u plone_buildout bin/buildout -Nv

      • got a lot of errors like the following.  I found that this page said these errors are harmless and that they can safely be ignored.

        Compiling /usr/local/Plone-4.3.3/buildout-cache/eggs/Products.ATContentTypes-2.1.14-py2.7.egg/Products/ATContentTypes/skins/ATContentTypes/getValidCriteriaForIndex.py
        SyntaxError: ("'return' outside function", ('/usr/local/Plone-4.3.3/buildout-cache/eggs/Products.ATContentTypes-2.1.14-py2.7.egg/Products/ATContentTypes/skins/ATContentTypes/getValidCriteriaForIndex.py', 10, None, 'return context.allowedCriteriaForField(index, display_list=True)\n'))

    • /etc/init.d/plone4_server_mschuh start
  • Plone Hotfix 20151208 (December 8, 2015) - followed buildout directions just like I did for the October 6, 2015 update described above.  Be careful to type in the egg line properly and don't mess with downloading the hotfix.  The buildout command downloads the hotfix.
  • Plone Hotfix 20160419 (April 19, 2016) - followed buildout directions just like I did for the October 6, 2015 update described above.  Be careful to type in the egg line properly and don't mess with downloading the hotfix.  The buildout command downloads the hotfix.
  • Plone Hotfix 20170117 (Jan 17, 2017) - On Nov 28, 2017, I found that the buildout.cfg had Jan 17 as the date and the diff on it and the previous version showed that the hotfix had been added.  No notes recorded at the time.
  • Plone Hotfix 20171128 (Nov 28, 2017)   
    • The repository is located in /opt/plone4.3.11-r1/zinstance
    • First did an rcs checkin and reset the permissions.  
    •  

      ci -l ./buildout.cfg
      chown plone_buildout.plone_group buildout.cfg
      chmod go+r buildout.cfg

       

    • Followed the Oct 6 directions
    • I found that the log message was in /opt/plone4.3.11-r1/zinstance/var/log/instance.log and had a message that showed that the hotfix had been installed.

Zope

  • Login at http://www.losaltosrobotics.org:8080/manage
  • Installing Zope talks about how to install Zope, login to manage it, resetting the admin password, and more topics.
  • Users and Security talks about user accounts and how to configure them.
  • There are two sets of users. One at the higher (or lower if you want to call it that) Zope level and another within Plone. Both have acl_users in their name. To get to the Zope set, go to http://www.losaltosrobotics.org:8080/manage and select acl_users in the left frame and then users in the main frame. To access the Plone set go to http://www.losaltosrobotics.org:8080/Main/acl_users/source_users/manage_users. I figure this out by reading through The Zope2 Book Users and Security chapter. The Zope users can edit the Plone Los Altos robotics content (/Main). I don't think the Plone users can do Zope management. Jonathan pointed out "that the Plone team was aiming for support of multiple websites, each with a different set of users, it makes some sense that they would" have multiple sets of users.

HTML

  • !-- closed out by -- is the tag for a comment.

Linux

  • The Linux Documentation Project
  • Debian GNU/Linux System Administrator's Manual (Obsolete Documentation)
  • Package tools
    • apt-get update - Updates the package list
    • apt-get search vim - Searches for packages with vim in the name
    • apt-get install wireless-tools - Installs wireless tools package and any dependencies
    • dpkg -L vim - List files installed with the vim package
    • dpkg -l - List the installed packages
    • apt-get upgrade - Installs the latest set of packages.
    • apt-get clear - clears cached copy of packages to free disk space.  Does not change what is installed.  Packages will be downloaded again if they are reinstalled.
    • flash-kernel  - This loads any new kernel formed onto the boot loader on the LosAltosRobotics  plug computer.
  • Handy Commands
    • Reset the timezone with:  dpkg-reconfigure tzdata
  • Wireless Networking
    • Wireless Tools for Linux  - command line tools.
      • iwlist eth1 scan - list available networks and print information about them
      • iwconfig eth1 or iwconfig - list wireless connections

Los Altos Robotics Server

  • The Los Altos Robotics Plone instance is backed up via cron by using the script in ~root/bin called backup_LosAltosRoboticsWebsitePloneDatabase. It uses the repozo.py script to make a copy in with the Plone website instance at /var/lib/zope2.10/instance/plone-site/backupOfLosAltosRoboticsWebsiteUsing_repozo and on the /backups/LosAlotosRobotics partition of the hard disk at /backups/LosAltosRobotics/backupOfLosAltosRoboticsWebsiteUsing_repozo. The repozo.py script is able to correctly make a copy of an open Data.fs file and make incremental backups of it. It can also be used to restore to any date. See the script for details. The output of the cron backups are kept in ~root/bin/backup_LosAltosRoboticsWebsitePloneDatabase.log.

 


Dave Lordemann's Notes

These come from emails that Dave sent Michael

  • The javascript references that do the email address hiding are not allowed through. (there are being filtered out as possibly un-safe) I changed the configuration to allow it. from the ZMI interface -> under "Main" click on "portal_transforms", then click on "safe_html", changed "remove_javascript" from a "1" to a "0" and then submit at the bottom. I THINK this should be a temporary change, until we have time to research how to do this in their view of safe practices.

Dave's Notes on turning on/off the self-registration/"Join"ing

On the Plone 3.3 system that we are using as of August 2010, we have decided to disable the feature that lets anyone join as a "Member". Having this enabled allowed SPAMers and others to place content on our server and then refer to it in a way to disguise that it was SPAM.

To disable self-registration on a Plone 3.3 system:

  • login to the system with an administrator account.
  • go to "http://www.losaltosrobotics.org:8080/Main/", in the upper right corner click on the "site setup" button. If you don't see "site setup" then you don't have administrator privileges
  • click on the "Security" button under the Plone Configuration section
  • check or un-check the box for "Enable self-registration"
    I believe that Plone must be restarted for this change to take effect. Use the "Maintenance" button under the Plone Configuration section. On the current system this may take 5 minutes or more for the restart of Plone.

 

Now, try to hide login and join so that only folks making changes are troubled with login and folks just looking for information are not confused into thinking they need to join.

  • go to "http://www.losaltosrobotics.org:8080/manage", in the left pane select "portal_actions: then "user"
  • click on "login", find the check box "Visible?" and remove the check and click the "Save Changes" button
  • click on "join", find the check box "Visible?" and remove the check and click the "Save Changes" button
  • I wasn't able to find the template for the you are not logged in reminder, but this isn't very important and the reminder doesn't show up. Maybe we did this previously and have just forgotten how we did it.

It is still possible to navigate to http://www.losaltosrobotics.org:8080/Main/join_form, but actual joining is disabled, and joining would require email to be sent to verify the email address. Since our server isn't setup to send email, that wont be possible.

these instructions are based on http://plone.org/documentation/kb/hide-not-logged-in-and-join-links, but some of the instructions don't look like our version of Plone, so read a little farther to get the alternate locations for things.

 

Jonathan Wilde's Notes

These come from emails that Jonathan sent Michael

  • In response to my question about if moving the site by copying the Data.fs file has any hidden problems, Jonathan said: "The one additional step that may be required for future upgrades (but not now) is upgrading the database schema. You can do this by going to `portal_migrations` in http://schuh.homelinux.com:8081/Main/manage and upgrade the database version after making a backup of the current database."
  • I asked about fixing the visual editor (WYSIWYG Editors for Plone)and he wrote "If you choose to use the visual editor in the future, you may run into issues with this. While some of the newer implementations of contentEditable (which all of the visual editors out there build on top of) are starting to become more robust, they often have issues with embedded scripting. An anchor tag surrounded with a heading tag might be less problematic while retaining the visibility of the button."
  • For the entire Plone 3.x series, you'll unfortunately need to also have Python 2.4 installed. I've found that Python 2.4 and Python 2.5 can coexist without issue. Just be careful to prepend python2.4 to the command line if you call any of Plone's scripts directly (the shebang at the top will probably direct them to the python executable, rather than the python2.4 executable).

    I looked inside one of the Plone dependency packages [1] and it looks like the pure Python code is being deposited in the /usr/share/pyshared/ directory, which should be accessible to Python 2.4 if you install the python2.4 package.

    Using the Plone version in the repositories looks like it should be fine. Just be careful to update the database version and follow all of the instructions (especially for upgrading the database version) in the Plone upgrade guide [2].

    The Python 2.4 dependency in Plone will be changed to a Python 2.5/2.6 dependency in Plone 4. Plone 4 was originally slated to come out on August 1, but no final release candidates have come out yet. Judging by the quantity of bugs in their bug tracker [3], I'd expect them to release a final version sometime in September or October.

    Hope this helps, Jonathan

    [1] http://packages.debian.org/lenny/all/python-openid/filelist
    [2] http://plone.org/documentation/manual/upgrade-guide
    [3] http://dev.plone.org/plone/query?status=assigned&status=new&status=reopened&group=status&milestone=4.0