Troubleshooting

Warning

These pages contain information intended for Bloch staff members. If you are not staff, you must not attempt anything described here regarding changing settings, adjusting hardware or calling on-call support numbers

  • If something is wrong but you don’t yet know what the cause is, a good starting point is to run the ‘Checkup’ macro. This is launched from the ‘ControlGUIs’ folder on either of the windows measurement computers, and will highlight any beamline components that are in an unusual state.

_images/checkup.png

  • If you still don’t know what the cause is, the ‘restart sardana’ shortcut on the beamline Linux computer desktop will restart the pool and the macroserver and can fix a variety problems.

  • A third angle of attack is to open Astor and look for device servers that are in fault or not running, but should be. Often a restart of the server within Astor is sufficient to fix the problem.

  • If you cannot open the front-end, check whether the hutch has been searched.

  • If the undulator is not responsive even after a sardana reset, or if you still cannot open the front end despite searching the hutch, it can be helpful to ask the operator if they can see any issues from their side. Use either the chat window or call them on 072 247 29 45.

On-call support options

Phone numbers are listed in this elogy post (blue/white network only)

Icepap

The most common icepap issues are motors powering off due to closed loop errors or due to overcurrent errors. In both cases the motor involved is powered down, and cannot be powered on again until the error is cleared.

Warning

Both error types often happen randomly, and do not indicate a real underlying problem. But if you ever have to do this multiple times within a short space of time, something is probably wrong and you should investigate (slipping encoder, something needs lubricating…)

Within icepapcms, the nature of the problem can be confirmed by invoking a console (you can can right click on the relevant rack in the left hand panel to do this directly, skipping the need to type in the rack address). Status information is obtained by (<axis number>:?VSTATUS) (e.g. for the Carving y-axis, 2:?VSTATUS).

Closed loop errors

This occurs when the encoder and the stepper fall out of sync (e.g. motor turns but encoder doesn’t increment). This can be fixed by either issuing an esync command from a spock terminal (ipap_esync <motor_sardana_name_or_tango_alias>) OR in icepapcms - disable closed loop mode, send config to board, enable closed loop mode, send config to board, attempt to power on again.

Overcurrent errors

These are more difficult to deal with. If you are within standard working hours, call KITOS for support. If not, the currently recommended procedure is:

  1. In icepapcms, power down all other motors in the same rack as the problem motor

  2. Issue a RESET command in the icepap console (RESET <rack number>)

  3. Power cycle the rack (physical switch at the back)

  4. It may be necessary to re-enable one or more axes by holding up the ‘enable’ toggle switch(es) on the front panel for a few seconds until the LED turns green.

  5. You should now be able to power on the motor again in icepcms

If you’re desparate for more information, good luck finding what you need in the icepap manual here.

SES

The interface connecting SES to the manipulator and to the monochromator relies on a certain process being alive on a remote virtual computer somewhere. If this goes down (as can happen during e.g. a power cut), the symptom will be that SES cannot read manipulator positions or monochromator energy, and the UI will feel extremely sluggish as it repeatedly attempts to poll these devices and times out waiting for a reply. You need to contact KITOS to bring the required service back. It may help to direct them to the bottom part of this elogy post.

Another common reason for SES becoming very sluggish is the number of regions in the current sequence file growing too long. The time SES needs to process the file increases nonlinearly with the number of entries, and can reach several seconds when there are 12+ regions. The fix is to delete unused regions, or start a new sequence file.