ccf
Wiki: Frequently Asked Questions

Edit this page | Links to this page | Page information | Attachments | Refresh page

 

CollabNet Connector Framework FAQ

This is the complete FAQ for the CollabNet Connector Framework. We have used our Wiki for this FAQ so that it is easy for you to help us maintain the contents of this FAQ. You only have to be a logged in project member to use the Wiki. There should be an Edit this page link displayed above.

Feel also free to join our mailing list and ask as many questions as you like.

We also like to point you to a nice FAQ related to openadaptor. Most of the questions answered there might be of interest for you as well since the CCF is based on openadaptor.

Return to Wiki FrontPage

Contents

  1. General Questions
    1. What is the license of CCF?
    2. Do I have to edit CCF's data base tables by hand?
    3. I am getting lost with the CCF data base configuration, what should I do?
    4. What are the systems currently supported by the CCF?
    5. I have configured my wiring but after I started it the connector just idles around, what's wrong?
    6. I got an unsatisfied link error while trying to use the HP QC plugin: "java.lang.UnsatisfiedLinkError: This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem." What should I do?
    7. When accessing a SFEE 4.4 instance, the CCF reports a 404 - Not Found error. What should I do?
    8. Do I always have to create a mapping for the technical ids of release field values in order to synchronize release fields?
    9. I initially imported lots of artifacts and now they all belong to the connector user. Should not they belong to the initial creator?
    10. I initially imported lots of artifacts and now their creation date is the import date. Should not it be the original creation date?
    11. When I create or update artifacts with the connector user/resynch user account, these updates will not show up immediately in the other system/attachments will not be synchronized. Why is that?
    12. How do I run CCF scenarios as a service?
    13. Where do I get the binaries to run CCF services with a 64 Bit Java VM?
    14. Why do most of the CCF components only support pulling artifacts and no "push mode"?
    15. What is the best polling interval for my scenario?
    16. How does CCF react on temporary network outages and invalid session keys?
    17. How can I enforce an immediate resynch for newly created artifacts?
    18. Are there other data bases but HSQLDB supported by the CCF?
    19. I like the CCF to also read artifacts lastly modified by the connector/resync user to export them? How can I do that?
    20. How can I turn off attachment shipping?
    21. How can I run two independent CCF instances on the same machine?
    22. CCF appears to hang and consumes a lot of CPU time, what went wrong?
    23. Can I move my CCF production instance to another machine?
    24. How can I install CCF GUI if I do not have Internet Access (disconnected install)?
    25. While trying to do an initial synch of many thousand TF artifacts, CCF only reports timed out operations or complains about insufficient Java Heap Size, how can I still import all my artifacts?
  2. Exception Handling Related Questions
    1. How to route and store exceptions?
    2. How to react on network timeout related exceptions?
    3. How to react on session fault related exceptions?
    4. How to react on exceptions caused within reader or processor components?
    5. How to react on exceptions caused by writer components?
    6. How to react on exceptions caused by core openAdaptor components?
    7. How does artifact replay work?
    8. Can I automatically replay certain kind of errors?
    9. Why are reprocessed artifact records not deleted automatically?
  3. Conflict Handling Related Questions
    1. What is a conflict?
    2. How are conflicts detected?
    3. How are conflicts resolved?
    4. Are there any plans to implemented more sophisticated conflict resolution?
    5. Do you have a concrete example how artifact replay behaves in a real world scenario?
  4. Security Related Questions
    1. How can I access SSL/TLS enabled systems with the CCF?
    2. Why do I need TDAdmin rights for the HP QC connector user?
    3. How should I protect CCF directories on my file system?
    4. Which ports do I have to protect from unauthorized access?
    5. Can I obfuscate the passwords/credentials stored in my .properties files?
    6. How do I secure my data base connection over SSL/TLS?
  5. Remote Control Related Questions
    1. What CCF features can currently controlled/observed remotely?
    2. How can I turn on the CCF JMX HTTP interface?
    3. Is it possible to expose further CCF functionality via JMX and to programmatically interact with the CCF?
    4. How can I register centrally which CCF instances are running?
    5. What I can do to improve CCF's GUI remote performance?
  6. XSLT related questions
    1. The Xpath function replace does not work for me, what can I do?
    2. I have mapped some QC fields that contain values but these are always transported as empty fields. What's wrong?
    3. Does CCF support graphical data mapping?
    4. How can I ignore all artifacts with an id <= X?
    5. How can I ignore newly created artifacts?
    6. How can I ignore artifact where the value of field "status" is "Closed" ?
  7. Requirement/Planning Folder related questions
    1. I get the QC Exception "Duplicate entity: ", what is wrong?
    2. Can I map multiple requirement types to TF planning folders?
    3. Can I map TF planning folders to multiple requirement types?
    4. If I map TF planning folders to multiple requirement types I get weird warning messages from the entity service. What's wrong?
    5. Does CCF support version control in combination with requirements?
    6. I get “Parent artifact <...> for artifact <...> is not yet created on the target system for combination <...>. Since a project mapping exists for <...> CCF bails out now” error message from the entity service, what's wrong?
    7. When I move a requirement in HP QC, it does not get moved immediately in the target system, what's wrong?
  8. Ignoring Quality Center Locks
    1. What is this feature?
    2. How do I tell CCF to ignore Quality Center Locks?
    3. How does the feature work?
      1. Note on Comment handling
    4. Example

General Questions

What is the license of CCF?

Starting from version 1.1.1, CCF has been licensed under the Apache 2.0 license. CCF relies on some third party components that have been licensed differently, see CCFDependencies for details.

Do I have to edit CCF's data base tables by hand?

No, we recommend that you exclusively use our GUI for this purpose.

I am getting lost with the CCF data base configuration, what should I do?

Typically, you never have to start from scratch when configuring the CCF data base. As long as you do not intend to connect to an unsupported system, you will always find a sample scenario and a related row in the sample CCF data base. If you are curious about the actual meanings of the different columns in that table, please refer to http://ccf.open.collab.net/wiki/SupportedSystems.

What are the systems currently supported by the CCF?

Take a look at http://ccf.open.collab.net/wiki/SupportedSystems

I have configured my wiring but after I started it the connector just idles around, what's wrong?

Two reasons are possible if your source system is not a HP QC system. First of all it might be that there are currently no updates available in your source system.

If it is a HP QC system, a second reason is possible. Every field that should be synchronized has to be configured that its changes are logged in the history.

I got an unsatisfied link error while trying to use the HP QC plugin: "java.lang.UnsatisfiedLinkError: This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem." What should I do?

Please download the Visual Studio 2005 libraries and install them. You need at least version 8.00.50727.762 from VC2005SP3 so please also update if you already have an older version of this library installed.

When accessing a SFEE 4.4 instance, the CCF reports a 404 - Not Found error. What should I do?

Appendix C of the SFEE administration guide misses one apache redirect rule that is necessary to access SFEE's web services.

Make sure that your httpd.conf contains this redirection rule:

RewriteRule ^/sf-soap44/(.*) http://localhost:8080/sf-soap44/$1 [P]

Do I always have to create a mapping for the technical ids of release field values in order to synchronize release fields?

Unfortunately, some APIs of CCF's supported system can only update release fields of tracker items if you know the technical release of your id. However, the reader and writer components of these systems typically provide a boolean property called "translateTechnicalReleaseId" that allows you to use the human readable name of the release instead of the technical id. This property is set to false in our default wirings since the lookup involves additional web service calls that might decrease the update performance. If you can live with these additional web service calls, you may edit the wiring of your integration scenario and set the "translateTechnicalReleaseIds" property to true.

I initially imported lots of artifacts and now they all belong to the connector user. Should not they belong to the initial creator?

The CCF uses the connector user (and sometimes the resynch user) to do all the synchronization work. Since CCF uses the web services of the target systems and these web services usually do not support to perform operations on behalf of another user, all artifacts will be created with the connector user as creator. The same applies to update operations as well.

I initially imported lots of artifacts and now their creation date is the import date. Should not it be the original creation date?

The web services, CCF uses to synchronize artifacts, usually do not support to set another creation date but the actual time when the artifact was imported into the target system.

When I create or update artifacts with the connector user/resynch user account, these updates will not show up immediately in the other system/attachments will not be synchronized. Why is that?

Please do not use the connector user/resynch user for manual updates. These accounts are reserved for the CCF that needs these users to differentiate whether the update has been made by itself or by a "real" user.

How do I run CCF scenarios as a service?

Take a look at http://ccf.open.collab.net/wiki/RunCCFAsAService

Where do I get the binaries to run CCF services with a 64 Bit Java VM?

Take a look at http://ccf.open.collab.net/ds/viewMessage.do?dsForumId=591&dsMessageId=398879

Why do most of the CCF components only support pulling artifacts and no "push mode"?

Take a look at http://ccf.open.collab.net/wiki/PushSupport

What is the best polling interval for my scenario?

Take a look at http://ccf.open.collab.net/wiki/PollingInterval

How does CCF react on temporary network outages and invalid session keys?

Take a look at http://ccf.open.collab.net/wiki/RetryReloginConfiguration

How can I enforce an immediate resynch for newly created artifacts?

Take a look at http://ccf.open.collab.net/wiki/Resynching

Are there other data bases but HSQLDB supported by the CCF?

Take a look at http://ccf.open.collab.net/wiki/OtherDatabase

I like the CCF to also read artifacts lastly modified by the connector/resync user to export them? How can I do that?

The CSFE, HP QC and CEE reader all feature an optional property called "ignoreConnectorUserUpdates". If you set this property to false, all comments, attachments and artifacts will be retrieved by the CCF.

How can I turn off attachment shipping?

All reader components have an optional property "shipAttachments". The default value for this property is true, but if you set it to false, no attachments will be shipped. Add the following line into the reader component configuration within your config.xml files to turn off attachment shipment:

<property name="shipAttachments" value="false"/>

How can I run two independent CCF instances on the same machine?

The only obstacle is in setting up a second hdsqldb database instance that runs in parallel with the first default hsqldb database instance. The database instance is defined in centralCCFDatabase/*. You may change the centralccfdatabaseconf and rename "xdb" into "xdb2" but it only changes the name of the database. A second DBMS requires to change the connection port by creating a new file server.properties. Into that file you need to write "server.port=9002" (the default port of hsqldb is 9001). After these changes the second hsqldb database instance can be started.

With the second hsqldb instance in place one can go to the connectors. In addition to the normal configuration you need to change the ccf.properties as well - replace the dbms url string from "ccf.db.url=jdbc:hsqldb:hsql://localhost/xdb" into "ccf.db.url=jdbc:hsqldb:hsql://localhost:9002/xdb2". After these changes the connectors will talk to the second hsqldb instance and the first CCF instance is left untouched of actions in the second CCF instance.

CCF appears to hang and consumes a lot of CPU time, what went wrong?

In 99 percent of the case, this behavior is triggered if you use Internet Explorer to access HP QC from the same machine you run CCF. Please make sure not to access HP QC using IE on the CCF machine.

Can I move my CCF production instance to another machine?

As long as the other machine is in the same time zone and has the same language settings, you just have to copy your CCF folder to the new machine. If the other machine is using another system language, there might be problems in replaying already quarantined artifacts, so make sure to replay artifacts before you do the migration. If you like to synchronize TF, SFEE or CSFE (does not apply to HP QC and CEE) and the new machine is running in another timezone, the last synchronization timestamp for your project mappings starting from TF will not be adjusted to the new time zone. Hence you have to use the "reset synchronization status" action in CCF GUI and reset this timestamp to the current date.

How can I install CCF GUI if I do not have Internet Access (disconnected install)?

If your company policy does not allow connecting to the CCF Eclipse Update site directly, create a local copy by exporting it from svn

svn export http://ccf.open.collab.net/svn/ccf/trunk/www/update-site

Then zip/tar the output and add it to Eclipse as a zipped local update site.

While trying to do an initial synch of many thousand TF artifacts, CCF only reports timed out operations or complains about insufficient Java Heap Size, how can I still import all my artifacts?

Please take a look at http://ccf.open.collab.net/wiki/SynchingMultipleThousandArtifactsAtOnce

Exception Handling Related Questions

How to route and store exceptions?

Since the CCF is based on openAdaptor, it uses openAdaptor's mechanisms to handle exceptions. Within the wiring configuration, one single exception handler, a chain of exception handlers or even a message hospital can be set up. Information how to setup these exception handlers can be found here. The example integration scenarios that come with the CCF typically use a form of the message hospital paradigm. All exceptions are stored with their time stamp, originating component, user name, error description and artifact data within a file or data base table. This permanent storage is called "hospital" in the openAdaptor jargon. Artifacts within the hospital can be analyzed to find out why the could not be shipped. Feel free to add further exception handlers to these scenarios just by changing the wiring configuration.

It is also possible to fix the content of failed artifacts and replay them in the openAdaptor chain. However, replaying artifacts often causes more trouble (stale updates, duplicate artifacts, out-of-sync data) than alternative approaches. The following sections give suggestions how to react on different kind of exceptions.

How to react on network timeout related exceptions?

Network timeout related exceptions will only occur if you have the retry feature of your reader and writer components turned off. Look into your RetryReloginConfiguration to see how you can turn this feature on. Since network timeout related exceptions are just temporary, you can just rerun (or keep running) the connector again to get rid of these problems. If only some artifacts have been affected by this behavior, you can do a mass update on them to force a resync. An artifact replay is not suggested.

How to react on session fault related exceptions?

Session fault related exceptions will only occur if you have the relogin feature of your reader and writer components turned off. Look into your RetryReloginConfiguration to see how you can turn this feature on. Since session fault related exceptions are just temporary, you can rerun (or keep running) the connector again to get rid of these problems. If only some artifacts have been affected by this behavior, you can do a mass update on them to force a resync. If the session was lost during a non-atomic operation it might happen that your target system contains some incomplete artifacts. If these incomplete artifacts have not been completed after a rerun but a new complete artifact has been created, just delete the incomplete one by hand. An artifact replay is not suggested.

How to react on exceptions caused within reader or processor components?

If a reader or processor component caused an exception, the synchronization status table, the reader component, the source system or the processor component has been configured in an improper way. In all cases the best idea is to fix the problem (e. g. wrong repository name in the synchronization status table(target values), wrong artifact schema in the source system, wrong property of the reader component, wrong XSLT file of the XSLTProcessor or wrong SQL statement for the ?EntityService/MappingDBUpdator) and rerun (or keep running) the connector. In most cases, fixing the problem will result in changing the affected artifacts. Therefore, just rerunning the connector (or keep it running) will typically solve the problem. An artifact replay is not suggested.

You should never change the SQL update statements of the entity service or the MappingDBUpdator if you are not absolutely sure what you are doing! Misconfiguration of these properties may result in an invalid synchronization status and identity mapping table which in turn may need a full resync of all artifacts within your repositories.

How to react on exceptions caused by writer components?

If a writer component throws a non-timeout or non-session fault related error, the error typically lies in the synchronization status table, the XSLT scripts of the XSLT processors or the artifact schema within the target system. Misconfiguration in the synchronization status table (target values) will typically result in an exception for every single artifact. In this case, changing the table and rerunning (or keep running) the connector will suffice. XSLT and schema related problems may just affect some artifacts. After having solved the root cause (look at the CCF error code), the affected artifacts should be refreshed/updated in the source system so that they are automatically reread by the reader component. Artifact replay may work for this scenario but should be avoided if the artifacts in the source system can be refreshed without causing major issues. If you like to do artifact replay, use the replay functionality in the CCF GUI.

How to react on exceptions caused by core openAdaptor components?

If an exception was caused by openAdaptor's main event loop or the data base breaks down, the CCF will shutdown itself in order not to corrupt any data. If the CCF binary has been wrapped by a watch dog (details here) it will be restarted again. Otherwise, you have to manually restart the CCF and it will recover from its crash. If the crash happened during a non-atomic operation it might happen that your target system contains some incomplete artifacts. If these incomplete artifacts have not been completed after a restart but a new complete artifact has been created, just delete the incomplete one by hand. An artifact replay is not suggested in this situation.

How does artifact replay work?

Some CCF scenarios use a data base based hospital. The table where the quarantined artifacts are stored is called HOSPITAL. Only scenarios using the data base based hospital do currently support artifact replays.

The hospital tables looks very similar to the identity mapping table but also contains all information about the exception and its location. Furthermore, there are two status fields called FIXED and REPROCESSED.

If you click the replay button in the CCF GUI, the reprocessed column value will be set to true and the responsible CCF scenario will try to process the quarantined artifact again.

If an error happens during replay, the artifact will not be marked as FIXED but as REPROCESSED and a new entry will show up in the hospital.

If the replay succeeds, the artifact will be deleted from the hospital. The CCF GUI will only replay data base rows that has not been reprocessed before (you may reopen replayed hospital entries though).

All bidirectional wirings involving two different systems (like TF and CEE) will now also use an extended version of the entity service. This extended version allows to check whether the currently processed artifacts is quarantined in the hospital and has not been reprocessed so far. It is configurable whether the entity service will skip the artifact if there is an unprocessed entry or will pass it if the source artifact has been changed again in the source system after the artifact was quarantined (default behavior). If the advanced properties of the entity service are not used the component will function as before.

Can I automatically replay certain kind of errors?

In CCF's default settings, only artifacts that have been marked to be replayed in the CCF GUI will be picked up again.

If you look into your config.xml file, you will find a property "sql" of the ?HospitalDatabaseExtractor with the following value:

SELECT ID, GENERIC_ARTIFACT, ORIGINATING_COMPONENT FROM HOSPITAL WHERE REPROCESSED = FALSE AND ERROR_CODE = 'replay' AND SOURCE_SYSTEM_ID = '?' and SOURCE_REPOSITORY_ID = '?' AND TARGET_SYSTEM_ID = '?' and TARGET_REPOSITORY_ID = '?' ORDER BY ID ASC LIMIT 1

This query is responsible to pick a quarantined artifact from the hospital.

To automatically replay artifacts with error code "objectLocked" and "objectCheckedOutError" as well, try the following one:

SELECT ID, GENERIC_ARTIFACT, ORIGINATING_COMPONENT FROM HOSPITAL WHERE REPROCESSED = FALSE AND (ERROR_CODE = 'replay' OR ERROR_CODE = 'objectLocked' OR ERROR_CODE = 'objectCheckedOutError') AND SOURCE_SYSTEM_ID = '?' and SOURCE_REPOSITORY_ID = '?' AND TARGET_SYSTEM_ID = '?' and TARGET_REPOSITORY_ID = '?' ORDER BY ID ASC LIMIT 1

If you only like to replay quarantined artifacts that are at least 5 minutes in the hospital, try this one:

SELECT ID, GENERIC_ARTIFACT, ORIGINATING_COMPONENT FROM HOSPITAL WHERE REPROCESSED = FALSE AND (ERROR_CODE = 'replay' OR ((ERROR_CODE = 'objectLocked' OR ERROR_CODE = 'objectCheckedOutError') AND DATEDIFF('mi', CONVERT(SUBSTRING(TIMESTAMP, 1, 19), TIMESTAMP), NOW) > 5)) AND SOURCE_SYSTEM_ID = '?' and SOURCE_REPOSITORY_ID = '?' AND TARGET_SYSTEM_ID = '?' and TARGET_REPOSITORY_ID = '?' ORDER BY ID ASC LIMIT 1

Why are reprocessed artifact records not deleted automatically?

Quarantined artifacts are only deleted automatically if its replay has been successful. Otherwise, they are kept to be able to still examine the original payload that caused the problem.

If you want to change that, you have to update the HospitalUpdatorSQLConverterReprocessed bean in your config.xml:

Change the script property from

var transactionId = oa_data.getRootElement().selectSingleNode("@transactionId").getText();

if (transactionId == "unknown") transactionId = "-1";

oa_data = "UPDATE HOSPITAL SET REPROCESSED = TRUE WHERE ID = '" + transactionId + "'";

to

var transactionId = oa_data.getRootElement().selectSingleNode("@transactionId").getText();

if (transactionId == "unknown") transactionId = "-1";

oa_data = "DELETE FROM HOSPITAL WHERE ID = '" + transactionId + "'";

If you do this, you will not be able to reopen quarantined artifacts in the GUI though.

Conflict Handling Related Questions

What is a conflict?

A conflict in the CCF is a situation where a ?GenericArtifact from a source repository should be written into the target repository and one of the following conditions turns out to be true:

  1. the exact same version of the artifact has been already synchronized with the target repository (in this case, we speak about a duplicate update)
  2. the target repository has been already updated with a newer version (that is higher version number) of the source artifact (in this case we speak about a deprecated update)
  3. The target artifact has been changed by a user since the last update by the CCF (this is the most typical conflict)

How are conflicts detected?

The CCF conflict detection mechanism is source and target system independent. It relies on the fact that every artifact from the source and the target system support the concept of a "version" and a last modification date. Whenever an artifact is changed within a system, the version is incremented. If two versions of one artifact are considered, the artifact version with the higher version number will be considered to be more up to date.

If a source or target system does not support versions, it can still participate in the artifact synchronization process but conflict detection is not fully supported (as long as the last modified date is still supported, changes that have not happened within the same second might still be detectable). We have thought about comparing the field values of artifacts in order to find out whether the artifact has been changed, but have not followed this idea since it does not detect changes where a field value "a" changes to "b" and to "a" again within one update cycle.

Conflicts from category 1 and 2 are sorted out by the ?EntityService component. The ?EntityService will look into the identity mapping table to find out the version number/modification date when the source artifact was lastly synchronized with the target artifact. If the current version number of the source artifact is not greater than this value and the last modification date is not greater or equal, the artifact update will be ignored.

If the duplicate/deprecated check was passed successfully, the ?EntityService will update the targetArtifactVersion top level attribute of the generic artifact with the version/last modified date of the target artifact when it was lastly synchronized by the CCF itself. These values are part of the identity mapping table as well. The writer component responsible for the target system will compare the current version/last modification date of the target artifact with these values. If the current values are greater than the values in the generic artifact, a conflict from category 3 has happened.

How are conflicts resolved?

Conflicts from category 1 and category 2 are always ignored and will not result in an update.

Once a conflict of category 3 is detected, the writer component responsible for the target system will have a look into the conflictResolutionPriority top level attribute of the generic artifact. The value of this attribute is typically determined by the conflictResolutionPriority column of the associated synchronization status table row for this particular synchronization. You can graphically change this value using our CCF GUI.

If the value is "alwaysIgnore", the artifact changes will not be updated. However, the synchronization status table will be updated. Typically, the synchronization status table will contain another row for the other synchronization direction that will contain the "alwaysOverride" policy so that in case of a conflict, the same system will always win.

If the value is "alwaysOverride", an update will happen and will override the changes made in the target system. If the synchronization status rows for both directions both contain the "alwaysOverride" policy, changes will be always overridden on the "first come, first serve" basis and the systems might temporarily contain different field values for the same artifact (if changes on the same artifact in both systems happen between the same update cycle).

If the value is "quarantineArtifact" no update will happen and the artifact will be quarantined in the hospital. You can inspect the artifact in the hospital to find out the conflicting change.

For mappings that target HP Quality Center, a fourth option, "alwaysOverrideAndIgnoreLocks", is available. Please see the section below for details.

If both synchronization status rows contain the "quarantineArtifact" policy, once a conflict is detected, the CCF will stop synchronizing the artifact in question and an administrator has to manually resolve the conflict (and update the identity mapping table). This is the safest configuration (in no case, any data will be overridden) but also the most work intense one.

The most typical configuration is the "alwaysOverride"/"alwaysIgnore" combination to clearly define a "leading system".

Are there any plans to implemented more sophisticated conflict resolution?

Deviations from the general conflict resolution strategies are currently discussed here and reported as action items here.

Do you have a concrete example how artifact replay behaves in a real world scenario?

Take a look at http://ccf.open.collab.net/wiki/ExceptionHandlingExample

Security Related Questions

How can I access SSL/TLS enabled systems with the CCF?

Take a look at http://ccf.open.collab.net/wiki/SecureAccess

Why do I need TDAdmin rights for the HP QC connector user?

HP QC's COM API only exposes a subset of the defect API through ordinary function calls. To retrieve all information necessary for a bidirectional integration, CCF has to use so called "commands" that allow to issue custom SQL statements via the COM API. These commands can only be executed by a user with the TDAdmin role.

How should I protect CCF directories on my file system?

Always ensure that only persons with admin rights for all systems that should be synchronized with the CCF have acceess to CCF's directories. These directories contain sensitive information like the credentials of the connector and resynch users. Other users should not have access to CCF configuration directories at all.

Which ports do I have to protect from unauthorized access?

CCF's data base is remotely accessible via port 9001, the JVM is accessible via JMX (scenario specific, see service wrapper .conf file for details) and JMX HTTP console (scenario specific see ccf.properties or CCF GUI for details). It is highly recommended to protect all these ports from authorized access or activate the DB / JMX authorization features (passwords/certificates).

Can I obfuscate the passwords/credentials stored in my .properties files?

Yes, starting from CCF 1.4.2, you can obfuscate your passwords. See the CCF Password Obfuscation Guide for details.

How do I secure my data base connection over SSL/TLS?

Please read the guide at http://www.hsqldb.org/doc/guide/ch07.html first. Currently, HSQL only supports server side authentication. To switch from unencrypted to encrypted data base access, add the following lines to your centralccfdatabase.conf:

wrapper.java.additional.2=-Djavax.net.ssl.keyStorePassword=<your keystore password>
wrapper.java.additional.3=-Djavax.net.ssl.keyStore=<path to your key store>

Please make sure that the certificate in your key store exactly matches the host name of your connection string, e. g. if your connection string is jdbc:hsqldb:hsqls://mycompany.com/xdb, the common name of your certificate has to be mycompany.com, "localhost" typically does not work. You also have to make sure that the trust store, CCF GUI is using, contains the certificate in question. See http://ccf.open.collab.net/wiki/SecureAccess for more details how to do this.

Remote Control Related Questions

What CCF features can currently controlled/observed remotely?

Currently, CCF supports to remotely change the following things:

  • mapped repositories: If the synchronization status table is modified with a graphical JDBC tool or the CCF GUI
  • restart the CCF: By using the CCF JMX interface (see CCF GUI), the openadaptor component can be used to shutdown the CCF which will trigger an immediate restart.
  • change the log level of every CCF component: By using the CCF JMX interface (see CCF GUI), the log4j component can be used to dynamically change the log level for every CCF component.
  • see the configuration and metrics of every CCF component: By using the CCF JMX interface (see CCF GUI), all CCF components will be exposed with its remotely controllable properties. If you select components ending with the term "metrics", you can also turn on data collection for every single component.
  • observe the memory consumption of the CCF: By using the CCF JMX interface (see CCF GUI), you can have a look at the current memory consumption of the CCF.

How can I turn on the CCF JMX HTTP interface?

Most of the CCF wirings already contain the necessary components to expose a JMX management console. If you start your wiring and still cannot access it, please add the following components to your config.xml file:

<bean class="org.openadaptor.core.jmx.MBeanServer">
        <description>
                Embed this in your config to expose jmx interface via http
                and rmi.
        </description>
        <constructor-arg value="8082" />
</bean>

<bean id="SystemUtil" class="org.openadaptor.util.SystemUtil">
        <description>
                Embed this in your config to expose useful management bean.
        </description>
</bean>

<bean id="LogAdmin"
        class="org.openadaptor.thirdparty.log4j.LogAdmin">
        <description>
                Embed this in your config to expose log4j management.
        </description>
</bean>

Is it possible to expose further CCF functionality via JMX and to programmatically interact with the CCF?

JMX can be used to programmatically control all exposed JMX properties of the CCF, see the Java JMX web site for details. If you need to access properties of the CCF that are not yet exposed via JMX, please tell us and we will implement the needed JMX interfaces for you.

How can I register centrally which CCF instances are running?

If you are interested in that feature, you may take a look at this solution.

What I can do to improve CCF's GUI remote performance?

The most band width consuming operation in CCF GUI is to retrieve the hospital entries and to check whether these are already outdated. If you do not need the info whether hospital entries are already outdated, you can dramatically improve CCF's GUI performance by turning the "Flag outdated" option under the "Hospital" section in CCF's GUI preferences off.

XSLT related questions

The Xpath function replace does not work for me, what can I do?

CCF is currently using the built in Java XSLT processor. This XSLT processor is not fully XSLT 2.0 compliant (it does not support the replace function for instance).

Therefore, you have to use some other functions like translate or ...-substring-...

Here is an example how you could strip "_admin" from a string (equivalent to <xsl:value-of select='replace(.,"_admin","")'/>)

        <xsl:variable name="userWithoutAdmin" select='substring-before(.,"_admin")'/>
        <xsl:choose>
                <xsl:when test='$userWithoutAdmin!=""'>
                        <xsl:value-of select='$userWithoutAdmin'/>
                </xsl:when>
                <xsl:otherwise>
                        <xsl:value-of select='.'/>
                </xsl:otherwise>
        </xsl:choose>

We have talked about replacing the internal XSLT processor in the JVM with Saxon (fully XSLT 2.0 compliant). However, this would break compatibility with our current XSLT scripts (because we use some Xalan specific ways to call Java functions from XSLT like the stripHTML method) so we decided not to do this until it becomes necessary.

I have mapped some QC fields that contain values but these are always transported as empty fields. What's wrong?

The following fields do not contain scalar values in QC but refer to release objects: BG_DETECTED_IN_REL, BG_DETECTED_IN_RCYC, BG_TARGET_REL, BG_TARGET_RCYC. In CCF versions smaller than 1.4.1 you can neither retrieve their values in CCF nor can you update them. Starting from version 1.4.1, these fields will contain the technical ids of the QC objects in question. You can also use these technical ids to set the field's values. Furthermore, there are read-only fields called BG_DETECTED_IN_REL_HUMAN_READABLE, BG_DETECTED_IN_RCYC_HUMAN_READABLE, BG_TARGET_REL_HUMAN_READABLE, BG_TARGET_RCYC_HUMAN_READABLE which expose the human readable display names of the objects in question. Since the human readable names are not unique across objects, you can only use technical release ids to create/update a defect.

Does CCF support graphical data mapping?

Starting from version 1.2.1, CCF supports graphical data mapping in combination with the CCF GUI and Altova MapForce. Further information can be found here.

How can I ignore all artifacts with an id <= X?

Whenever your XSLT script sets the generic artifact top level attribute "artifactAction" to "ignore" nothing will be created/updated in the target system.

Here is an XSLT snippet that changes the artifactAction so that only artifacts starting after the artifact stored in the "lastArtifactNotSynchronized" variable are synchronized. In this example, this would be artifact "{urn:devproj_1.0.0}issue:ccf320"

Please adjust this to your own settings. You also have to adjust the artifact prefix stored in the "artifactPrefix "variable. In this example, the artifact prefix is "{urn:devproj_1.0.0}issue:ccf"

<xsl:template match='/ccf:artifact[@artifactType = "plainArtifact"]'>
<artifact xmlns="http://ccf.open.collab.net/GenericArtifactV1.0">
<xsl:copy-of select="@*" />
                        
<!-- Only synchronize artifacts that have been created after lastArtifactNotSynchronized -->
<xsl:variable name="lastArtifactNotSynchronized" as="xs:string">{urn:devproj_1.0.0}issue:ccf320</xsl:variable>

<xsl:variable name="artifactPrefix" as="xs:string">{urn:devproj_1.0.0}issue:ccf</xsl:variable>

<xsl:variable name="sourceArtifactIdWithoutPrefix" as="xs:string" select="substring-after($sourceArtifactId, $artifactPrefix)"/>

<xsl:variable name="lastArtifactNotSynchronizedWithoutPrefix" as="xs:string" select="substring-after($lastArtifactNotSynchronized,$artifactPrefix)"/>

<xsl:if test="number($sourceArtifactIdWithoutPrefix) &lt;= number($lastArtifactNotSynchronizedWithoutPrefix)">
        <xsl:attribute name="artifactAction">ignore</xsl:attribute>
</xsl:if>
<xsl:apply-templates />
</artifact>
</xsl:template>

How can I ignore newly created artifacts?

Sometimes you do not want to create new artifacts by CCF at all but only want to update existing ones.

Whenever your XSLT script sets the generic artifact top level attribute "artifactAction" to "ignore" nothing will be created/updated in the target system.

Here is an XSLT snippet that changes the artifactAction so that only artifacts that have been already created in the target systems will be synchronized. This is done by checking whether the original value of this attribute has been set to "create". In this case, the artifact should be created in the target system (so it does not exist already).

<xsl:template match='/ccf:artifact[@artifactType = "plainArtifact"]'>
<artifact xmlns="http://ccf.open.collab.net/GenericArtifactV1.0">
<xsl:copy-of select="@*" />
                        
<xsl:if test="@artifactAction = 'create'">
        <xsl:attribute name="artifactAction">ignore</xsl:attribute>
</xsl:if>
<xsl:apply-templates />
</artifact>
</xsl:template>

How can I ignore artifact where the value of field "status" is "Closed" ?

Whenever your XSLT script sets the generic artifact top level attribute "artifactAction" to "ignore" nothing will be created/updated in the target system.

Here is an XSLT snippet that changes the artifactAction so that artifacts whose status field with name "status" is set to "foo" will be ignored.

Please adjust this to your own settings if your field name is different or you like to filter on a different field value.

<xsl:template match='/ccf:artifact[@artifactType = "plainArtifact"]'>
<xsl:variable name="statusFieldValue" as="xs:string" select="/ccf:artifact/ccf:fi​eld​[@fieldName='status']"/>
<artifact xmlns="http://ccf.open.collab.net/GenericArtifactV1.0">
<xsl:copy-of select="@*" />
                        
<xsl:if test="$statusFieldValue = 'Closed'">
        <xsl:attribute name="artifactAction">ignore</xsl:attribute>
</xsl:if>
<xsl:apply-templates />
</artifact>
</xsl:template>

Requirement/Planning Folder related questions

I get the QC Exception "Duplicate entity: ", what is wrong?

The names of QC requirements have to be unique across the requirement type. If CCF tries to update/create a second entity with the same title, this exception will occur. Please choose a different name.

Can I map multiple requirement types to TF planning folders?

Yes, you can, just use the CCF GUI wizard, create a HP QC requirement type to TF PF mapping and repeat this step for every requirement type. A requirement without a parent will be created as top level planning folder in TF. If the requirement has a parent, it will be created as child planning folder under the planning folder that represents the parent requirement in TF.

Can I map TF planning folders to multiple requirement types?

Yes, you can, but you have to carefully read the following explanations: If you map TF planning folders to multiple HP QC requirements, the entity service has to determine for every single project mapping whether to update or create an artifact. If the TF planning folder has been created by CCF, the identity mapping table already contains an entry for the reverse direction (TF PF to requirement type) as well. Consequently, the entity service will correctly identify the corresponding requirement and will do the update. Unfortunately, this will not be the case for all other TF PF to requirement type mappings: Since no mapping is present, new requirements would be created. To prevent this from happening, you may create an XSLT rule that only passes artifact updates and resynch requests but no creation or deletion requests.

If I map TF planning folders to multiple requirement types I get weird warning messages from the entity service. What's wrong?

If you receive messages like "There was no prior identity mapping for the resync request ... in the identity mapping table even after the entity service waited for ... milliseconds. Discarding the artifact." in combination with multiple project mappings from TF planning folders to requirements, you do not have to worry: If you look at the project mappings you have setup from TF to QC, you will probably find out that you have mapped the TF planning folders to multiple QC requirement types. This step has been necessary because TF planning folders have been created from different QC requirement types. Consequently, the corresponding QC requirement for a given TF PF is only present for one of those project mappings. For all other project mappings, the entity service cannot find any corresponding artifact on the QC side and will discard the artifact since the resynch option only makes sense in combination with an already created artifact. Please make sure that only PF with the resynch or update artifact action get transported to QC.

Does CCF support version control in combination with requirements?

If CCF detects that version control is enabled for HP QC requirements, it tries to checkout requirements before performing update operations. If the requirement has already been checked out by another user, CCF will quarantine the artifact (error code: objectCheckedOutError), so that you can replay quarantined artifacts later. You could also define a rule to automatically replay artifacts of this error code. For bidirectional synchronizations with heavy parallel use on both sides it might be easier to turn version control off if you do not rely on the base lining features of HP QC.

I get “Parent artifact <...> for artifact <...> is not yet created on the target system for combination <...>. Since a project mapping exists for <...> CCF bails out now” error message from the entity service, what's wrong?

This error message tells you that the parent artifact (e. g. parent QC requirement) of the currently shipped artifact (e. g. QC requirement) has not been synchronized so far. If the parent artifact had been part of an entity type (e. g. QC requirement type), you have not mapped with CCF, you would have only received a warning and the parent/child relationship would have been ignored. This error message however tells you that the parent artifact should have been synchronized before. If it has not, this is an indication that it

  • was not picked up by CCF yet. This is especially the fact if the parent artifact's number is 0 which is the requirement id of the QC top level folder called "Requirements". In this case, just touch the parent artifact again to let CCF pick it up (make sure you have history logging enabled in QC).
  • is currently in the hospital (typical reasons would be wrong data mapping, missing permissions in the target system or a missing parent artifact)
  • is already in CCF's update queue but has not been processed yet. See the BestPractices wiki page how you can avoid this situation. In this case, you just have to wait until the artifact comes through.

In all cases, once the parent artifact has been created, you can just replay the failed child artifact shipment from the hospital. It could be that you have to repeat this step because a whole hierarchy has to be synchronized.

When I move a requirement in HP QC, it does not get moved immediately in the target system, what's wrong?

CCF only detects changes in HP QC if a field has been changed that is logged in the audit table (history flag is enabled). If you move a requirement around, the only field that changes is the last modification time stamp, so please make sure that the last modification field "RQ_VTS" has the history flag set to true.

Ignoring Quality Center Locks

What is this feature?

Since version 1.4.1, CCF supports a new conflict resolution policy, alwaysOverrideAndIgnoreLocks, for mappings to HP Quality Center. As the name suggests, the basic policy is the same as for the alwaysOverride policy described in the section on conflict resolution above.

When the alwaysOverride policy tries to write an artifact that is currently locked in Quality Center, the artifact is quarantined. Since sometimes users may hold a lock on an artifact for an extended period of time, this can result in delayed shipment of changes to artifacts. The new conflict resolution policy enables CCF to temporarily take control over locks in Quality Center in order to update an artifact. The locks are restored to the original owner after the shipment is completed.

How do I tell CCF to ignore Quality Center Locks?

You can instruct CCF to ignore locks in Quality Center by setting the conflict resolution policy to alwaysOverrideAndIgnoreLocks in CCF GUI for mappings targeting Quality Center as shown in the CCF GUI documentation.

How does the feature work?

When the conflict resolution policy is set to alwaysOverrideAndIgnoreLocks, CCF performs the following steps when shipping an artifact to Quality Center:

  1. Check whether the artifact is locked.
  2. If it is locked, take ownership of the lock.
  3. Ship the artifact. CCF will only apply changes to fields and attachments, comments are not updated in this case but stored in the hospital for later processing.
  4. Restore the previous lock.

If the artifact is changed in Quality Center after step 3, Quality Center will override the changes from the data in the shipment. However, Quality Center will only update the field values that the user editing the artifact actually touched, meaning, if the user only touched unrelated fields, changes will effectively be merged.

Note on Comment handling

Comments are handled specially, because Quality Center handles all comments of an artifact as a single HTML text field. If a user added a comment after CCF shipped the artifact in step three, comments added by CCF would be overwritten. Instead, CCF makes sure that comments are only added once the object is unlocked by temporarily storing them in the hospital with the error code set to "objectLocked" and periodically replaying the comments after a configurable amount of time, until the object is unlocked.

Example

Assume a fictious artifact with the following fields:

Field

Field Content

title

title

priority

1-Low

description

the description

comment

a comment

The user locks opens the artifact in Quality Center, changes the title to "QC title", the priority to "3-Medium", adds QCatt.txt as an attachment and adds a comment "QC comment". However, he does not yet close the artifact because he wants to discuss the change with a colleague.

Meanwhile, another user modifies the corresponding artifact in (e.g.) ?TeamForge and sets the title to "TF title", updates the description to read "TF description", adds a comment "TF comment" and adds attach.txt as an attachment.

CCF will take ownership of the lock and apply the TF user's changes:

Field

Field Content

title

TF title

priority

1-Low

description

TF description

comment

a comment

attachment

attach.txt

Since the object is was locked by the Quality Center user, the comment is stored in the hospital for later processing. As the last step, CCF restores the Quality Center user's lock on the object.

Now, the Quality Center user returns from his discussion and closes the artifact, thus releasing the lock and applying the changes:

Field

Field Content

title

QC title

priority

3-Medium

description

TF description

comment

a comment, QC comment

attachment

attach.txt, QCatt.txt

Later, CCF retries the postponed comment and finds that the artifact is no longer locked. Thus, it adds the comment, resulting in the final state:

Field

Field Content

title

QC title

priority

3-Medium

description

TF description

comment

a comment, QC comment, TF comment

attachment

attach.txt, QCatt.txt

Frequently Asked Questions (last edited 2011-05-13 12:30:20 -0800 by jonico)