How about a post just to say what you're up to now days? Or maybe the continued pursuit of the perfect desk! lol

Luckily, I don't have to do any Sharepoint coding (and I sincerely hope I never will), but Jake on Sharepoint would still be better than codestore rusting away ...

Awaiting the next post.

Although I was able to fetch all attachments, I did find it ambiguous to understand the main difference of the whereabouts of all embedded objects.

Another thing one has to keep in mind in a Java agent is that in order to not run out of memory one has to recycle what you are no longer needing.

you can't hide forever. We do miss you.

Just read your post. I did a near identical setup as yours for the large data migration (over 5 million documents, from multiple Lotus Notes databases & forms) about 5 years ago, from Lotus Notes to IBM DB2.

As part of that migration we also migrated & sync'd data records from a number of MS Excel spreadsheets, MS access databases, Lotus Approach databases, Lotus 123 spreadsheets and MS Word tables.

Had the same issues with the RichText migration. We chose to automatically print all the Notes documents to PDF files (to retain the format) and then just migrated the unformatted text.

We had many issues, as we were migrating multiple Lotus Notes databases (built by multiple dev's, over numerous years) and other data sources, which required a lot of data cleansing reports for missing fields, incorrect data types, etc...

Also, given that we moving to a new system (IBM Websphere and DB2), there was a lot of work required in data mapping between the old and new systems. We had to provide mechanisms for the SME's to do this mapping, with the results being used to for the final migration to the new system.

Feel free to give me a shout some time and we can compare notes. I may have a few things, tips, idea's; which may help you.

Regards

Patrick

Domino's problem has never been the data store nor connectivity, but always IBM not being able to push and actively promote its strong points. They probably never understood their own product, so they opted to puzzle their customers as much as possible.

The most recent masterpiece of marketing: IBM Domino Social Edition OpenSocial component V9.0.1

Doesn't that make you want to HAVE that product NOW? No? Me neither. It would have taken a third "social" in there to really rock my socks off.

There are tools to automatically convert from Notes to SharePoint, my employer has used them on a large scale.

That said, a low cost Notes 2 SQL solution could definitely be interesting. As Jerry suggested, why not try it? Start small and see if it takes of. Don't forget to market it though.

The email I'd like to receive would start something like "Hi. We have a legacy Notes system we need to get data out of. Can you help?"

Instead provide the source code with the tool so that users can customise it to their needs, and make this your selling point. You could add license conditions stopping people redistributing your code.

Speed was never really a factor though. The target SQL tables are just temporary containers for the (E)xported data, from where it's (T)ransformed before (L)oading in to the target platform (ETL).

As for Rich Text, I've not had to deal with that yet, luckily.

There is C-magic to extract the file from those, but afaik code to extract the file depends on the OLE object type.

In step2, I would save the docunid as a char field and let the sql-db generate a LongInt PK. Numbers are much faster in SQL than char.

Embedded Objects are hard.

@Keyth:

- Multivalue fields maybe saved as their own m-n joined table.

- For RichText I once parsed their content as html with jakarta-http-client from Domino. Nsf was running on Domino Server and I wrote a loop which fired http-Get requests for all documents with richText.

kind regards

Axel

A suggested feature might be making it target system agnostic. You could do this with MSADO as your data layer with simple connection string changes between MSSQL, DB2 and ORACLE.