IBM Connections Docs CR1 installation may fail with an error on Conversion server

It has happened to me in at least 3 different systems that the installation on Docs CR1 fails throwing an error about DocsConversion failed. Looking into the iFixInstall.log file you will see

…………………
2016-07-22 12:13:28,415 DEBUG WASX7209I: Connected to process “dmgr” on node connectionsCellManager using SOAP connector; The type of process is: DeploymentManager

WASX7303I: The following options are passed to the scripting environment and are available as arguments that are stored in the argv variable: “[checkin_json, conversion-config.json]”2016-07-22 12:13:28,415 INFO DocsConversion update failed!
2016-07-22 12:13:28,415 DEBUG Traceback (most recent call last):
File “applypatch.py”, line 31, in update_product
update_apps()
File “applypatch.py”, line 67, in update_apps
update_conversion_binary()
File “applypatch.py”, line 168, in update_conversion_binary
result = CONFIG.call_was_task(‘remote_install_a_version’,[me_hostname, escapes_for_was(str(target_list)), str(sym_count)],True)
File “c:\IBMConnectionsDocs_2.0_CR1\config.py”, line 60, in call_was_task
was_out = self.call_wsadmin(args)
File “c:\IBMConnectionsDocs_2.0_CR1\config.py”, line 43, in call_wsadmin
raise Exception(“Exception thrown while executing WebSphere command:” + ws_out)
Exception: Exception thrown while executing WebSphere command:WASX7209I: Connected to process “dmgr” on node connectionsCellManager using SOAP connector; The type of process is: DeploymentManager

WASX7303I: The following options are passed to the scripting environment and are available as arguments that are stored in the argv variable: “[remote_install_a_version, DOCS.yourdomain.com, [u\’DOCS.yourdomain.com\’], 8]”

WASX7017E: Exception received while running file “../execwas.py”; exception information: com.ibm.websphere.management.exception.AdminException: CWWSY0102E: Target with name DOCS.yourdomain.com was not found.

The error is somewhat misleading because upon reading it I immediately went on my system and verified that a target in the job manager with the correct name existed. So this is not the real cause.

I opened a PMR and today had a remote session with David McCarthy from IBM Support in Dublin and Jie Peng  from the Docs Dev labs in China, and they solved the issue.

  1. Go in the directory where you have the CR1 installation software
    – Edit the file applypatch.py
    – comment those 2 lines in red
    def update_apps():
    “””update apps in present work directory”””
    for filename in os.listdir(‘./’):
    if(filename.endswith(“.ear”) or filename.endswith(“.ear.zip”)):
    appname = get_app_name(filename)
    CONFIG.call_was_task(‘update_app’,[appname, filename])
    #if os.path.exists(“docs_remote_installer.zip”):
    #update_conversion_binary()
    – go in the DocsApp subdirectory and delete the file concord-config.json
    – run applypatch again
  2. Stop IBM Conversion Cluster, Websphere Console->Servers->Clusters->WebSphere application server clusters->IBMConversionCluster->Stop
  3. In the  directory where you have the CR1 installation software, open the DocsConversion subdirectory. Unzip the file docs_remote_installer.zip in a directory of your choice, for example c:\temp\docs_remote_installer
  4. Change directory to c:\temp\docs_remote_installer\installer\ and run the following command
    upgrade_node.bat – -installroot [CONVERSION_INSTALL_ROOT] – -symcount
    [SYM_COUNT]
    where [CONVERSION_INSTALL_ROOT] value you can get from the ISC (Environment -> WebSphere Variables-> CONVERSION_INSTALL_ROOT
    [SYM_COUNT] is the number of symphony instances, you can get this by counting how many inst* in [CONVERSION_INSTALL_ROOT]\symphony, usually it is 4 or 8.for example: upgrade_node.bat – -installroot c:\IBM\ConnectionsDocs\Conversion – -symcount 8
  5. Check fixpack.log in directory [CONVERSION_INSTALL_ROOT]\logs\
  6. Start IBMConversionCluster, Websphere Console->Servers->Clusters->WebSphere application server clusters->IBMConversionCluster->Start

Now open a browser and go to http://yourdocsserver/conversion/version, you should see something like this

{“timestamp”:”20160711-0948″,”version”:”2.0.0.1″,”build_description”:”IBM Connections Docs 2.0.0″,”product_name”:”IBM Connections Docs”}

If you still see version 2.0.0 then go back in the directory where you have the CR1 installation software, then in the DocsApp subdirectory and delete again concord-config.json. Run again the applypatch command, that will fix it.


Integrate IBM Connections Docs with BOX

This is a very useful integration, once set up you will be able to use IBM Docs to edit your files stored in BOX. What I found is that is not so easy to set up, requires some fiddling with json files and this can be tricky, at least for me it was since I am not that familiar with json. But with some attention in editing the file it can be done.

IBM published in June 2016 a Technote on how to perform the integration. https://www-304.ibm.com/support/docview.wss?uid=swg21981293, and I followed the instructions written there. The first part is pretty much straightforward, just do what is indicated there and you will create your BOX application for Docs integration.

1. Log in to the Box developer console at https://developers.box.com/
  2. Create a Box application, choose a name, for example, "Edit in IBM Docs".
  (Make sure Content API Access Only is selected.)
  3. Remember client_id and client_secret for the Box application.
  You need to import the OAuth credentials to the IBM Docs system.
  4. The redirect_uri in the OAuth2driverscallback".
  5. Set the user type to Standard Box users.
  6. Go to the Web App Integrations section and click
  Create a New Web App Integration.
  7. Configure the web app integration:
     a. Choose a name for the web application, and fill in the description
        for the application.
     b. Add supported extensions for Docs: .docx, .pptx, .xlsx, .doc, .xls,
       .ppt, .ods, .odt, .odp, .csv, .txt.
     c. Under Permissions requirement, select Full permissions are required.
     d. In the Scoped to field, select
        The parent folder of the file/folder from which this integration is invoked.
     e. Set the category to Editing.
     f. Set the file type category to Documents.
     g. Set the integration status to Online.
     h. Make the integration open a popup in a new tab.
     i. Use REST method.
     j. Leave the Preliminary Callback URL field empty.
     k. The client callback URL should be the callback URL of the
        IBM Docs non-IBM product integration and must use the https protocol,
        such as "https://docs_server/docs/driverscallback".
     l. Callback parameters should be configured as follows:

GET:file_id:#file_id#
 GET:repository:rest
 GET:code:#auth_code#

8. Leave the other fields as default.

9. Save the web app integration and then save the application.

Then there are two options, depending if you are integrating when installing Docs or if you want to integrate an existing installation. I choose the second, since I already have some servers with Connections and Docs installed.

1. Download and install IBM Connections Docs 2.0 iFix 004 or IBM Connections Docs 2.0 CR1
2. Update the Editor Application configuration file <WAS install root>/profiles/Dmgr01/config/cells//IBMDocs-config/concord-config.json.

Now you have to add some code, paying close attention to the json file format.

(This is where I struggled a bit. You have to make sure you use the correct syntax and I strongly suggest you to use a smart editor like Notepad++ that will show the format of the file better than the simple Notepad)

You have to add this at the beginning of the file

  "x-frame-options":
     {
       "allow_option":"ALLOW-FROM",
       "allow_uri":"https://app.box.com"     } 

Then you have to update Update ExternalRestRepository configuration:

  {

           "id" : "external.rest",

           "class" : "com.ibm.docs.repository.external.rest.ExternalRestRepository",
             "config" :
             {
             "s2s_method" : "oauth2",
             "customer_id" : "box.com",
             "oauth2_endpoint" : "https://app.box.com/api/oauth2/token", 
             "j2c_alias" : "",
             "s2s_token" : "",
             "token_key" : "",
             "onbehalf_header" : "",
                          "media_meta_url" : "https://api.box.com/2.0/files/{ID}",
             "media_get_url" : "https://api.box.com/2.0/files/{ID}/content",
             "media_set_url" : "https://upload.box.com/api/2.0/files/{ID}/content",
             "docs_callback_endpoint" : "https://<docs_server name>/docs/driverscallback",
             "repository_home" : "https://app.box.com"
           }
        }                       

Update ExternalAuth configuration:

{
          "id" : "external.rest",
          "class":"com.ibm.docs.authentication.filters.ExternalAuth",
          "config" : {
          "s2s_method" : "oauth2"                                
          }                        
   }       

Update ExternalDirectory configuration, and add new keys “bypass_sso” and “current_user_profiles_url”.

{

            "id": "external.rest",
              "class": "com.ibm.docs.directory.external.ExternalDirectory",

            "profiles_url": "https://api.box.com/2.0/users/{ID}", 
              "current_user_profiles_url":"https://api.box.com/2.0/users/me",
              "bypass_sso":"true",
              "token_key": "",
              "docs_callback_endpoint": "https://<docs_server name>/docs/driverscallback",
              "oauth2_endpoint": "https://app.box.com/api/oauth2/token",
              "j2c_alias": "",
              "onbehalf_header": "",
              "customer_id": "box.com",
              "config": {
              "s2s_token": "",
              "s2s_method": "oauth2",
              "keys": {
              "org_id_key": "org_id",
              "photo_url_key": "",
              "url_query_key": "ID",
              "display_name_key": "name",
              "name_key": "name",
              "id_key": "id",
              "email_key": "login"
             }
         }
    }

Basically search for external.rest in the file and you will find these 3 sections.

Then Update socialConfig configuration

"socialConfig":{
  "profile": "null",
  "BIZCard": "null",
  "ST": "null", 
  "externalprofile":{
  "url": "https://api.box.com/2.0/users/{ID}",
  "useInitialName": false 
  }

Now sync all the nodes to propagate the changes from Deployment Manager to App Servers

There are two more steps:
Step 1: Set up OAuth2 credentials for Box
Extract the file customer_credential_mgr.py from the IBM Docs installation package, for example, CN30NML.zip\IBMConnectionsDocsrepo\ native\DocsApp_2.0.0.zip\installer\docs\tasks\
Run wsadmin.bat -lang jython -username xx -password xx -f ${PATH}/customer_credential_mgr.py -action add -customer customer_id -key key -value “value”
Where :
customer_id is the value specified in customer_id in ${WAS_INSTALL_ROOT}/profiles/AppSrv1/config/cells/{cell}/IBMDocs-config/concord-config.json.
Note: the technote says the value is the one specified in external_customer_id but this is wrong, use the one specified in customer_id
key could be oauth2_client_id or oauth2_client_secret
value is the value of the client_id or client_secret you should have copied when creating the BOX app. If you did not, you can always edit the BOX app and you’ll find the values you need
oauth
For example:
wsadmin.bat -lang jython -user xx -password xx -f customer_credential_mgr.py -action
add -customer box.com -key oauth2_client_id -value “l7xxf61984f99f404575a781d47c6bfebdca”

Step2: Add box.com https certificates into the WebSphere local trust store
a. Log into the WebSphere Application Server Administrative Console.
b. Expand Security and click SSL certificate and key management.
c. Under Configuration settings, click Manage endpoint security configurations.
d. Select the appropriate outbound configuration to get to the (cell) management scope.
e. Under Related Items, click Key stores and certificates.
f. Click the CellDefaultTrustStore key store.
g. Under Additional Properties, click Signer certificates > Retrieve From Port.
h. In the Host field, enter the api.box.com in the host name field, the port_number value 443 in the Port field, and the api.box.com in the Alias field.
i. Click Retrieve Signer Information.
j. Verify that the certificate information is for a certificate that you can trust.
k. Click Apply and Save.
l. Restart Deployment Manager, all nodes and app servers.

Now open a browser and log on to Connections, then in another tab log on to BOX, select a file from the list in BOX and click the small rectangle with the three dots
2016-07-21_18-53-34
A popup menu will appear
2016-07-21_18-55-04
Click on “More Actions” and you should see the BOX app you created before available
Image1
A popup will appear the first time
2016-07-21_19-01-09
Click Okay and a new tab will open with the document opened in Docs
2016-07-21_19-03-46

I would like to thank Wei Li and my friend Stefano Pogliani, both from IBM, for the help they provided me with.


And now for something completely different: how to recover several hundreds of MB of space on your iOS device.

On the web somewhere, I can’t recall where, I found this trick to recover space on your iOS device. Check how much space you’ve left in Settings-> General, in my case I was left with 800 MB. Now open the iTunes Store, go in movies and select a movie to rent greater than the space you have free, let’s say 4GB. Select to rent it, after a few seconds you’ll get the message that you don’t have enough space and told to manage the free space and you’ll be sent again in the settings. Check the free space you have now, it should be increased by several hundred MBs; in my case went from 800 MB to 1.2 GB. Repeat this again four or five times, until you see that the free space doesn’t increase anymore. In my case I ended up with 2.1 GB free.

I do not know exactly what iOS does to free the space, I guess it will clean some cache or the like, but I would have never been able to clean 1.3 GB of free space manually myself.


IBM Connections Docs 2.0 CR1 fix does not install

I tried to install Docs 2.0 CR1 but when I ran the command to perform the upgrade the task failed, specifically when updating the Conversion server, with the error “Target with name ic55.eld.it was not found.”

I quickly found out that two other esteemed friends, Christoph Stoettner and Milan Matejic, had encountered the same issue and were able to solve it.

The problem is in the file config.py that is in the patch installation directory. At the bottom of the file there the line “self.me_hostname = socket.gethostname()”. The solution  is to change that to “self.me_hostname = socket.getfqdn()”.

Once I did that, CR1 installed flawlessly. Thanks a lot to Christian and Milan!