IBM Verse on Premises 1.0.0.1 integration with Connections Files. How to make it work – Updated

I installed Verse on Premises 1.0.0.1 in my environment but when I tried to add a link to a file in Connections when composing an email I got an error “Sorry we cannot retrieve your files at the moment”.

After a chat with my excellent friend and IBM Champion Christoph Stoettner, he found the problem and the solution.
The official documentation about integrating VoP with Connections says to add a few lines at the end of the httpd.conf file:

• Make a backup copy of httpd.conf.
• Remove any comment symbols (#) from the following lines:
  LoadModule headers_module modules/mod_headers.so
LoadModule rewrite_module modules/mod_rewrite.so
• Find the following two lines:
  LoadModule was_ap22_module /opt/IBM/WebSphere/Plugins/bin/64bits/mod_was_ap22_http.so
WebSpherePluginConfig /opt/IBM/WebSphere/Plugins/config/webserver1/plugin-cfg.xml

  Copy the following lines and paste them directly after the two lines:

  RewriteEngine on
  
  # Minor change to adjust for Cloud vs On-Premises API variation of parameter name
  RewriteCond %{REQUEST_METHOD} PUT
  RewriteCond %{QUERY_STRING} ^(.*)uid=(.*)
  RewriteRule ^/profiles/photo.do /profiles/photo.do?%1userid=%2 [L]
  
  # Added necessary CORS headers when Origin header present
  Header unset Access-Control-Allow-Origin
  SetEnvIf Origin "^https://(vop_server_hostname\.)?(domain_name)$" origin_is=$0
  Header always set Access-Control-Allow-Origin %{origin_is}e env=origin_is
  Header always set Access-Control-Allow-Credentials "true" env=origin_is
  Header always set Access-Control-Allow-Headers "X-Requested-With, Content-Type, slug" env=origin_is
  Header always set Access-Control-Allow-Methods "POST, GET, OPTIONS, DELETE, PUT" env=origin_is
  
  Header always set Access-Control-Max-Age "1000" env=origin_is
  Header always set Access-Control-Allow-Methods "POST, GET, OPTIONS, DELETE, PUT" env=origin_is
  # Header always set Access-Control-Allow-Headers "X-Requested-With, Content-Type, Origin, Authorization, Accept, Client-Security-Token, Accept-Encoding, slug" env=origin_is
  Header always set Access-Control-Allow-Headers "X-Requested-With, Cache-Control, Content-Language, Content-Type, Expires, Last-Modified, Pragma, slug, X-Update-Nonce" env=origin_is
  Header always set Access-Control-Expose-Headers "Content-Disposition, Content-Encoding, Content-Length, Date, Transfer-Encoding, Vary, ETag, Set-Cookie, Location, Connection, X-UA-Compatible, X-LConn-Auth, X-LConn-UserId" env=origin_is
  
  # Added a rewrite to respond with a 200 SUCCESS on every OPTIONS request.
  RewriteCond %{REQUEST_METHOD} OPTIONS
  RewriteRule .* - [R=200,L]
  
  # Remove the Origin header if it exists for other requests (POST, GET, DELETE, PUT). Causes problems with Connections returning 403 response.
  RequestHeader unset Origin env=origin_is

   You have to change this line

  SetEnvIf Origin "^https://(vop_server_hostname\.)?(domain_name)$" origin_is=$0

  putting your server name and domain name.
  Examples
  vop_server_hostname = mail01
  domain_name = company.com

  `SetEnvIf Origin “^https://(mail01\.)?(company\.com)$” origin_is=$0`

This is not enough.
Since in the httpd.conf I use a virtual host, as is common in Connections, you need to add those lines also in the virtual host definition, so I copied them after the line

SSLProtocolDisable SSLv2 SSLv3

Restarted the http server and now the integration works.

UPDATE

The excellent Tim Clark, a good friend of mine and an IBM Champion, made a couple of interesting and useful suggestions in his comment to this post. I will put them here because not everyone read the comments

1. It is possible to make the https optional with the following RegEx, http(s)?
   So it all looks like this in the end.
   `SetEnvIf Origin “^http(s)?://(mail01\.)?(company.com)$” origin_is=$0`
2. you can put all the lines to copy and paste in a separate file so that you don’t have to have it listed twice in your httpd.conf file and then use an include to insret it (**twice**) where you need it.`Include “X:\IBM\HTTPServer\conf\vop.conf”`

Want a discount for attending IBM Connect ?

Stealing the post from my good friend Sharon Bellamy James. I couldn’t say this better.

IBM Champion for 2017

I had the great honor of being selected as IBM Champion for 2017, is my second year in a row.
Is great to be part of this group, and I would like to thank Amanda Bauman and Libby Ingrassia from IBM who look after us and do a great job for the Champions community.

There are 7 Champions from Italy, up from 5 of 2016. You can find the complete list here
https://www.ibm.com/developerworks/community/blogs/762e655e-e86c-4624-9662-ee81c6874de1/entry/Introducing_the_2017_IBM_Champions?lang=en

Error with database when editing with IBM Docs

On a server where I installed Docs I could not edit any file; whenever I tried to open in Docs a file I received an error “service is not available” in the browser and in Docs server SystemOut.log I see this:

000000f3 DocumentEdito E   CLFAD1003E: error occurs when getting editors from database                                 com.ibm.db2.jcc.am.SqlSyntaxErrorException: “CONCORDDB.DOCEDITORS.LEAVESESSION” is not valid in the context where it is used.. SQLCODE=-206, SQLSTATE=42703, DRIVER=4.19.26

I asked to the Docs guru, my friend Martti Garden, and I discovered that also on one of his servers he had the same error; he fixed it and told me what the solution was.
I forgot to run the updateschema command after applying CR1. By default when you install Docs and run the updateschema command, it brings the schema of the CONCORD db to 24. But when you apply CR1 you have to run the version of the command that comes with it and that brings the schema to 26.
After running the command all problems were gone and I can edit files in Docs.

Error when upgrading Connections 5.5 with CCM to CR2

While upgrading FileNet CE Server I got an error

ERROR occurred while installing FileNet applications
————————————————————
Traceback (most recent call last):
File “C:\IBM\Connections\lib\ccm.py”, line 400, in map_to_web_servers
ce.map_to_servers(self.web_servers)
File “C:\IBM\Connections\lib\wasobj.py”, line 1735, in map_to_servers
for m in self.installed_modules():
File “C:\IBM\Connections\lib\wasobj.py”, line 1726, in installed_modules
text = AdminApp.view(self.name, [‘-MapModulesToServers’])
com.ibm.ws.scripting.ScriptingException: com.ibm.ws.scripting.ScriptingException: com.ibm.websphere.management.exception.ConnectorException

org.apache.soap.SOAPException: [SOAPException: faultCode=SOAP-ENV:Client; msg=Read timed out; targetException=java.net.SocketTimeoutException: Read timed out]

————————————————————

Looking at it you can see is a SOAP timeout error.

I edited the file soap.client.props which is in C:\IBM\WebSphere\AppServer\profiles\Dmgr01\properties and changed the value of com.ibm.SOAP.requestTimeout from 180 (which is the default) to 0 (which means no timeout).

After the change I retried the update and everything worked fine.

Many thanks to Michael Urspringer for the suggestion.

I'll be speaking at SUTOL 2016 in Prague

I will be speaking at SUTOL 2016, the Czech User Group meeting which is in November 10-11 in Prague.

I will have two sessions:
– IBM Docs 2.0 from zero to hero, where I will demo a live installation of IBM Docs 2.0
– A Hitchhiker’s guide to troubleshooting Connections, together with the wonderful lady geek Sharon Bellamy-James

I have been before twice to Prague and I love the city, the availability of good beer is a plus :-), so I can’t wait to be there and meet some of my fellow IBM Champions and other friends.

IBM extends support for Notes/Domino until 2021 at least.

IBM has issued a press release announcing it will extend support for IBM® Notes®, IBM Domino®, IBM Notes Traveler, and IBM Enterprise Integrator for version 9 to September 2021.

Now, as my friend Gab Davis said in her blog post here, this does NOT mean support will end in 2021. I also agree the wording could have been better phrased using “at least”, but as Gab said there are IBM lawyers involved in this so things are not always as we would like them to be. Since I am not an IBM lawyer, I have said “at least until 2021” in this post title because since the press release does not mention an end of support, is a reasonable assumption to believe that support will be extended beyond that date.

I believe this statement from IBM is a good thing, it puts an end to the “Domino is dead” rumors that have been spreading around in these days. Why has that happened (again… this is the umpteenth time I heard that phrase, and has always been shown false) ? Because IBM is moving away from a major/minor release number schema in favor of a method of feature release more similar to what happens in the cloud space. How many customers care, or even know, what version is the software they are using in the cloud ? No one. All it matters is that from time to time new features are added and bugs are fixed. From now on IBM will release “Feature packs” that will enhance the functionalities of Notes and Domino, not caring anymore to call that Domino 9.0.2, 9.0.3, 9.1 or 10 or whatever. It will simply be Notes and Domino 9 updated to the latest features set available. So the fact that there will not be a new 9.x release does not means at all that Domino is dead or not evolving.

Domino is alive and evolving as it has been since R1.

Integrate IBM Connections Docs with BOX

This is a very useful integration, once set up you will be able to use IBM Docs to edit your files stored in BOX. What I found is that is not so easy to set up, requires some fiddling with json files and this can be tricky, at least for me it was since I am not that familiar with json. But with some attention in editing the file it can be done.

IBM published in June 2016 a Technote on how to perform the integration. https://www-304.ibm.com/support/docview.wss?uid=swg21981293, and I followed the instructions written there. The first part is pretty much straightforward, just do what is indicated there and you will create your BOX application for Docs integration.

1. Log in to the Box developer console at https://developers.box.com/
  2. Create a Box application, choose a name, for example, "Edit in IBM Docs".
  (Make sure Content API Access Only is selected.)
  3. Remember client_id and client_secret for the Box application.
  You need to import the OAuth credentials to the IBM Docs system.
  4. The redirect_uri in the OAuth2driverscallback".
  5. Set the user type to Standard Box users.
  6. Go to the Web App Integrations section and click
  Create a New Web App Integration.
  7. Configure the web app integration:
     a. Choose a name for the web application, and fill in the description
        for the application.
     b. Add supported extensions for Docs: .docx, .pptx, .xlsx, .doc, .xls,
       .ppt, .ods, .odt, .odp, .csv, .txt.
     c. Under Permissions requirement, select Full permissions are required.
     d. In the Scoped to field, select
        The parent folder of the file/folder from which this integration is invoked.
     e. Set the category to Editing.
     f. Set the file type category to Documents.
     g. Set the integration status to Online.
     h. Make the integration open a popup in a new tab.
     i. Use REST method.
     j. Leave the Preliminary Callback URL field empty.
     k. The client callback URL should be the callback URL of the
        IBM Docs non-IBM product integration and must use the https protocol,
        such as "https://docs_server/docs/driverscallback".
     l. Callback parameters should be configured as follows:

GET:file_id:#file_id#
 GET:repository:rest
 GET:code:#auth_code#

8. Leave the other fields as default.

9. Save the web app integration and then save the application.

Then there are two options, depending if you are integrating when installing Docs or if you want to integrate an existing installation. I choose the second, since I already have some servers with Connections and Docs installed.

1. Download and install IBM Connections Docs 2.0 iFix 004 or IBM Connections Docs 2.0 CR1
2. Update the Editor Application configuration file <WAS install root>/profiles/Dmgr01/config/cells//IBMDocs-config/concord-config.json.

Now you have to add some code, paying close attention to the json file format.

(This is where I struggled a bit. You have to make sure you use the correct syntax and I strongly suggest you to use a smart editor like Notepad++ that will show the format of the file better than the simple Notepad)

You have to add this at the beginning of the file

  "x-frame-options":
     {
       "allow_option":"ALLOW-FROM",
       "allow_uri":"https://app.box.com"     } 

Then you have to update Update ExternalRestRepository configuration:

  {

           "id" : "external.rest",

           "class" : "com.ibm.docs.repository.external.rest.ExternalRestRepository",
             "config" :
             {
             "s2s_method" : "oauth2",
             "customer_id" : "box.com",
             "oauth2_endpoint" : "https://app.box.com/api/oauth2/token", 
             "j2c_alias" : "",
             "s2s_token" : "",
             "token_key" : "",
             "onbehalf_header" : "",
                          "media_meta_url" : "https://api.box.com/2.0/files/{ID}",
             "media_get_url" : "https://api.box.com/2.0/files/{ID}/content",
             "media_set_url" : "https://upload.box.com/api/2.0/files/{ID}/content",
             "docs_callback_endpoint" : "https://<docs_server name>/docs/driverscallback",
             "repository_home" : "https://app.box.com"
           }
        }                       

Update ExternalAuth configuration:

{
          "id" : "external.rest",
          "class":"com.ibm.docs.authentication.filters.ExternalAuth",
          "config" : {
          "s2s_method" : "oauth2"                                
          }                        
   }       

Update ExternalDirectory configuration, and add new keys “bypass_sso” and “current_user_profiles_url”.

{

            "id": "external.rest",
              "class": "com.ibm.docs.directory.external.ExternalDirectory",

            "profiles_url": "https://api.box.com/2.0/users/{ID}", 
              "current_user_profiles_url":"https://api.box.com/2.0/users/me",
              "bypass_sso":"true",
              "token_key": "",
              "docs_callback_endpoint": "https://<docs_server name>/docs/driverscallback",
              "oauth2_endpoint": "https://app.box.com/api/oauth2/token",
              "j2c_alias": "",
              "onbehalf_header": "",
              "customer_id": "box.com",
              "config": {
              "s2s_token": "",
              "s2s_method": "oauth2",
              "keys": {
              "org_id_key": "org_id",
              "photo_url_key": "",
              "url_query_key": "ID",
              "display_name_key": "name",
              "name_key": "name",
              "id_key": "id",
              "email_key": "login"
             }
         }
    }

Basically search for external.rest in the file and you will find these 3 sections.

Then Update socialConfig configuration

"socialConfig":{
  "profile": "null",
  "BIZCard": "null",
  "ST": "null", 
  "externalprofile":{
  "url": "https://api.box.com/2.0/users/{ID}",
  "useInitialName": false 
  }

Now sync all the nodes to propagate the changes from Deployment Manager to App Servers

There are two more steps:
Step 1: Set up OAuth2 credentials for Box
Extract the file customer_credential_mgr.py from the IBM Docs installation package, for example, CN30NML.zip\IBMConnectionsDocsrepo\ native\DocsApp_2.0.0.zip\installer\docs\tasks\
Run wsadmin.bat -lang jython -username xx -password xx -f ${PATH}/customer_credential_mgr.py -action add -customer customer_id -key key -value “value”
Where :
customer_id is the value specified in customer_id in ${WAS_INSTALL_ROOT}/profiles/AppSrv1/config/cells/{cell}/IBMDocs-config/concord-config.json.
Note: the technote says the value is the one specified in external_customer_id but this is wrong, use the one specified in customer_id
key could be oauth2_client_id or oauth2_client_secret
value is the value of the client_id or client_secret you should have copied when creating the BOX app. If you did not, you can always edit the BOX app and you’ll find the values you need

For example:
wsadmin.bat -lang jython -user xx -password xx -f customer_credential_mgr.py -action
add -customer box.com -key oauth2_client_id -value “l7xxf61984f99f404575a781d47c6bfebdca”

Step2: Add box.com https certificates into the WebSphere local trust store
a. Log into the WebSphere Application Server Administrative Console.
b. Expand Security and click SSL certificate and key management.
c. Under Configuration settings, click Manage endpoint security configurations.
d. Select the appropriate outbound configuration to get to the (cell) management scope.
e. Under Related Items, click Key stores and certificates.
f. Click the CellDefaultTrustStore key store.
g. Under Additional Properties, click Signer certificates > Retrieve From Port.
h. In the Host field, enter the api.box.com in the host name field, the port_number value 443 in the Port field, and the api.box.com in the Alias field.
i. Click Retrieve Signer Information.
j. Verify that the certificate information is for a certificate that you can trust.
k. Click Apply and Save.
l. Restart Deployment Manager, all nodes and app servers.

Now open a browser and log on to Connections, then in another tab log on to BOX, select a file from the list in BOX and click the small rectangle with the three dots

A popup menu will appear

Click on “More Actions” and you should see the BOX app you created before available

A popup will appear the first time

Click Okay and a new tab will open with the document opened in Docs

I would like to thank Wei Li and my friend Stefano Pogliani, both from IBM, for the help they provided me with.