Compíling in Mono 1.9 + MonoDevelop

Hi!
First, let me say I am pretty impressed with Dataphor capabilities and I wish this project the best of luck
I would like to compile Dataphor in Mono 1.9 using MonoDevelop 1.0 the Dataphor.sln seems to open okey, and I can compile everything but Dataphoria.

But now I would like to test it and see if it works.

Can you provide me with some guidance on how to create my a "console version" of Dataphoria? Something like http://www.firebirdsql.org/manual/es/qsg15-es-creating.html but for Dataphor?

(I don't want anything fancy, I just want to be able to tell Dataphor to use a particular underlying databse for storage, and be able to issue D4 commands an see the results in simple text.)

Are you planning to offer something similar? I't would be pretty handy to certain administrative task on remote servers, (a much enhanced version could even integrate with WindowsPowershell, but right now, I would just like the minimal thing that can be done)

Thanks

Regards,
LuxSpes

D4Runner

Hi LuxSpes,

Thank you for your interest and welcome to the community!

I'm glad to hear the solution compiles for you on Mono, but I'm not surprised Dataphoria doesn't work, it has some third party dependencies that we are currently working on removing. The good news is, you don't need it, there is a fully-functional command-line tool called D4Runner that can be used to interact with the Dataphor server. You'll need to do at least the following to get it going:

1) Compile the Libraries solution (\Libraries\Libraries.sln). You may run into some dependency issues here as well, this is where all the connectors are located for different databases.
2) Open the Synergism solution (\Dataphor\Synergism.sln) and compile at least the D4Runner project. This solution also contains the service host and configuration utility, as well as the Web Client. I'd be interested to know how much of this compiles in Mono as well.

Once you have a D4Runner executable, you should be able to run an in-process instance.

Good luck, and if you have any further questions, please don't hesitate to ask!

Regards,
Bryn Rhodes
Alphora

No documentation on D4Runner

Hi!
I tried to find a guide on how to use D4Runner in DataphorSet.pdf, but I can not find any single mention of D4Runner...
I also tried looking for D4Runner in the Wiki, the only page I could find was one that I already knew (http://dataphor.org/index.php?title=Development:Source_Code) and it has not instructions on how to use D4Runner... and google wasn't helpful either...

I just would like to create Memory Device, create a Table on it, insert some rows, and the query them... (I'll be trying to do it, but without any documentation I am not sure I'll be successful)

Any guidance you can provide on the use of D4Runner would be really helpful...

Thanks
Regards,
LuxSpes

D4Runner Docs

I apologize for the lack of documentation on D4Runner. I threw together some docs on the wiki that should help you get started:

http://www.dataphor.com/index.php?title=Documentation:D4Runner

I'm not sure where the Mono runtime will resolve the ProgramData and AppData directories but they should contain a ServerAliases.config file with the alias definitions in them. You should be able to run an in-process instance as described in the documentation link above. Let me know if you run into any problems.

Regards,
Bryn Rhodes
Alphora

Comments in Wiki Discussion Page

Hi!
I wrote some questions and coments about the initial D4Runner documentation in:

http://www.dataphor.com/index.php?title=Talk:Documentation:D4Runner

Is that the proper place to write that? or should I have posted that in this forum?

Thanks,
Regards,
Luxspes

Proper placement

We're just making this up as we go, so I'm not sure there is a "right" place pe se. Off the top of my head, however, I would think that if the matter might be useful to somebody wanting to know more about the wiki topic, then an entry in the discussion page might be appropriate. On the other hand, the topic may get more exposure here. In fact, email notification is temporarily disabled on the wiki, so it is likely that nobody will know about a new discussion there at this point.

Best,

--
Nathan Allan [Alphora]

But to see replies in forum one needs to be logged in

Hi!
I thought that in fact the topic would get more public exposure wiki
because in this forum one can not read a reply to a post unless one is logged in, but anyone can look at the wiki without logging in... mmmm...

So... do you recommend I copy, paste and re-post here what I wrote in the wiki discussion page?

thanks,
regards,
LuxSpes

D4 Documentation Discussion

Hi LuxSpes,

I think the talk page should be fine, especially because the content being discussed is available there. I'll jump over there and try to answer the questions.

Regards,
Bryn Rhodes
Alphora

The requested feature is not implemented.

Hi!
Thanks for answering!
Now it seems to be loading the files, but afther that I am getting a "The requested feature is not implemented." messages (more details in the wiki discussion page)
Regards,
LuxSpes

Possible show stopper: Remoting support in Mono

Hi!
It seems Mono support for remoting may not be complete enough (see discussion page in wiki for more information):

System.NotImplementedException: The requested feature is not implemented.
at System.Runtime.Remoting.RemotingConfiguration.get_CustomErrorsMode () [0x00000]
at Alphora.Dataphor.DAE.Server.ServerHost..ctor (Alphora.Dataphor.DAE.Server.Server AServer, Int32 APortNumber) [0x00000]
at Alphora.Dataphor.DAE.Client.ServerConnection..ctor (Alphora.Dataphor.DAE.Client.ServerAlias AServerAlias, Boolean AAutoStart) [0x00000]

Any hints or suggested workarounds ?

Regards,
LuxSpes

CustomErrorsMode

Hi LuxSpes,

As I recall, the CustomErrorsMode had to be set in order to properly allow the remoting of non-system exceptions, which could be important as all exceptions coming from Dataphor fit this description. This may or may not be an issue for Mono; if they don't support the setting then one wonders what the "default" behavior is. Regardless, I agree that for now it should be noted and commented out.

Best,

--
Nathan Allan [Alphora]

Skipping first Show Stopper... getting another one!

Hi!
Well, I decided to comment out the line that calls RemotingConfiguration.CustomErrorsMode in the constructor of Alphora.Dataphor.DAE.Server.ServerHost and see what happens...

Now I get an exception saying it needs System.Data.SqlServerCe... why could that be? (see the discussion page for details)

Maybe I need to tell it to use a Memory Device?

Thanks
Regards,
LuxSpes

SQL CE... uh oh.

Had I been thinking I would have seen this coming. About a year ago, maybe more, we changed the storage for the Dataphor system catalog (compiled data dictionary for all registered libraries) from using D4 files into a table based catalog store. We picked SQL CE, which provided the desired table scan interface. This layer is abstracted so that it can be replaced, but this may be more than you bargained for.

You might be able to get the Windows client to work, depending on the rigor of Mono's Windows Forms support, but the server will require that the system catalog be replaced.

Sorry I didn't spot this obvious road block sooner. Some of our first efforts are to replace 3rd party dependencies from Dataphor. Perhaps we can consider SQL CE as such and switch it out for PostgreSQL.

--
Nathan Allan [Alphora]

Looking for ways of getting rid of SQL CE

Hi!

Okey, there is a hard dependency between Dataphor and SQL CE... lets see what can I do:

  • First question: Why did you change the storage for the Dataphor system catalog from using D4 files into a table based catalog store? wouldn't it be better if D4 was used? In that way the underlying database could easily be changed by changing the storage device... am I right? It would also be a way to show that Dataphor eats its own dogfood
  • PostgreSQL might work, but since: Dataphor is a stand-alone DBMS since it has a crude storage engine and Dataphor is a virtual DBMS in the sense that it uses other DBMS to provide storage management shouldn't we prefer to use Dataphor itself for this job? Another possible option could be an embedded database like, for example SharpSql. I think PostgreSQL might be too big and complicated to be used as a replacement for SqlCE, specially if used only for internal configuration (like the data dictionary) and I still think that D4 itself should be used to make it easier to switch storage engines.
  • And finally you say that This layer is abstracted so that it can be replaced that is fine with me, where can I find this layer? just tell me which one of the subprojects in which solution I have to use as the base for my code (since it is abstracted it should be possible for me to create a component and plug it into Dataphor... after that... Is there an option to configure the main class of the layer in one of Dataphor .config files ? or how is the alternative layer plugged in?

Thanks,
Regards,
LuxSpes

Database Engine

How about SQLite?. It seems to fit the bill well.

Here is an open source .NET wrapper:
http://sqlite.phxsoftware.com/

--
Nathan Allan [Alphora]

More on our catalog change

Adding to what Bryn said, we also liked the idea of having a non-memory table store intrinsically tied to the Dataphor Server so that we could move towards better scalability in places where we currently buffer sets in memory. The change to a persistent catalog store forced us to iron out all sorts of dependency issues in the catalog and has significantly improved the stability of the catalog. Ultimately we would like Dataphor to have it's own storage engine, but not to the exclusion of the storage integration architecture. We've built a few pieces of a storage engine already, but haven't had the time to finish it up.

Best,

--
Nathan Allan [Alphora]

SQL CE

Hi LuxSpes,

  • The catalog storage was changed from D4 files to achieve performance and transactional reliability. We could have used a D4 device to do this, but it would have brought so much of the SQL-device infrastructure into the DAE that it would have undermined the Storage Integration Architecture. It's a question of bootstrapping, we couldn't load the catalog from a storage device because we needed the catalog to describe the storage device. So we opted to use a dedicated catalog device with a layer that could be switched out if necessary. However, this is not a configurable aspect of the software, more on that later.
  • Dataphor's storage engine is still too crude to provide the performance and reliability we were after for the catalog. I agree that PostgreSQL probably has too big a footprint to be used effectively. Something like SharpHSQL may be a good fit, but we would need to make sure the license was compatible with the Dataphor license.
  • As far as the abstraction that enables this, there are two components: 1) The connection wrapper, and 2) The storage wrapper. The connection wrapper is the same connection wrapper that the rest of the DAE uses. The DAE.Connection namespace defines a SQLCEConnection that is used to wrap the connectivity aspects of CE. Any replacement would need to have a connection wrapper built for it. If the system already has an ADO.NET provider, this is a trivial exercise. As far as the storage wrapper, there is a SimpleSQLStore in the CatalogDevice folder of the DAE project. This store defines the behavior expected for a catalog backing store, and a new one would need to be provided to wrap whatever storage backend is used. Additionally, the CE dependency is hard-wired, so either an infrastructure for configuration would need to be added, or a compiler define used to switch to the different storage engine.

I hope this is enough to get you started, and if there is anything we can help you with along the way, please don't hesitate to ask.

Regards,
Bryn Rhodes
Alphora

Hi!

Thanks a lot!
I'll try it later and post here my comments!
Regards,
LuxSpes

Trying to compile D4Runner in Mono

Hi!
Thanks for answering.
I opened Libraries.sln, and I was unable to compile:
DAE.Connection.ADO (it requires adodb.dll obviously not available in Linux)
DAE.Connection.DataDirect.Oracle since I do not have theDataDirect providers.
DAE.Connection.DB2 and DAEDeviceDB2(I do not have IBM.Data.DB2.dll)
DAE.Connection.DB2400 (IBM.Data.DB2.iSeries.dll)
DAE.Connection.MySQL (EIDMySql.dll)
DAE.Connection.Oracle.Oracle (Oracle.DataAccess.dll)
DocSamples (Alphora.Dataphor.DAE.Connection.dll... it seems each of the "Connection" projects compiles to file with this same name
"Alphora.Dataphor.DAE.Connection.dll" but I do not understand why it is unable to see them... should I alter the paths and redirect them to for example /Libraries/MSSQLDevice/Alphora.Dataphor.DAE.Connection.dll)

Then I opened Synergism.sln and it all completed successfully! Now I'll try to run D4Runner...

Thanks again
Regards,
LuxSpes

Connection Assembly

Thank you for the feedback about the Mono compile. It sounds like apart from some dependencies, it compiled alright. The DocSamples problem is somewhat puzzling. The Connection assembly is part of the core Dataphor solution, and should be in the Dataphor/bin directory along with most of the other assemblies. The references from the Libraries should resolve automatically, but I'm wondering if the DocSamples one did not resolve correctly. You could try repointing the reference path to /Dataphor/bin and see if it picks up the core connection assembly.

Hope this helps,
Bryn Rhodes
Alphora

Which project generates Alphora.Dataphor.DAE.Connection.dll ?

Hi!
I do not have my linux computer with me right now, but I think remember that the reference in DocSamples was correctly pointing to /Dataphor/bin/Alphora.Dataphor.DAE.Connection.dll but there was no Alphora.Dataphor.DAE.Connection.dll file in /Dataphor/bin/...
Which subproject generates Alphora.Dataphor.DAE.Connection.dll on compilation?
(I mean, I know all the subprojects with the general pattern name DAE.Connection.* generate their own Alphora.Dataphor.DAE.Connection.dll but... which one of them (or not one of them) should be sending that Alphora.Dataphor.DAE.Connection.dll to /Dataphor/bin/ ? )
Thanks
Regards,
LuxSpes

Post build steps

I think that important library assemblies such as Connection are copied to \Bin using a post build step. I'll bet this copy command doesn't work properly in Linux for obvious shell command incompatibility reasons. I'm not sure if there is a different build configuration or something we can use to vary the post build step syntax, but for now you might try manually copying those files that have post build steps to bin.

Thank you for your efforts BTW.

--
Nathan Allan [Alphora]

BTW thanks for Dataphor

Hi!
BTW thanks for OpenSourcing Dataphor!
Regards,
LuxSpes

You're welcome

Thank you, we are very excited to dig Dataphor out of the closet, let it get more exposure and let more folks tinker under the hood. This is very exciting; but new to us so thanks for your patience.

--
Nathan Allan [Alphora]

DAE.Connection

Hi,

The project that generates that assembly is DAE.Connection. You did not list it as one of the ones that didn't compile, but it does reference the OleDbClient. Does the Mono runtime include the OleDbClient? Did this project compile successfully?

Regards,
Bryn Rhodes
Alphora

I do not see the OleDbClient

Hi!
I do not see the DAE.Connection.OleDbClient... do you mean DAE.Connection.ADO?
(It is already listed as one of those that do not compile in my previous post... it doesn't compile because it requires adodb.dll and it is not available in Linux)

Here are the contents of the Libraries folder... as you can see there is no folder with OleDbClient it its name:

ADOConnection MySQLDevice SASDevice
Connection ODBCConnection Security
DataDirectOracleConnection OracleConnection SimpleDevice
DB2400Connection OracleDevice SQLDevice
DB2Connection Sample.Common System.Integration
DB2Device Sample.Components System.Internet
Documentation Sample.Demo System.Platform
Frontend Sample.DetailAdd TestFramework
Libraries.sln Sample.Forms TestFramework.Controls
Libraries.userprefs Sample.Forum TestFramework.Coverage
Libraries.usertasks Sample.IssueTracker TestFramework.Coverage.Base
LinterDevice Sample.Pubs TestFramework.Coverage.Devices
MSOracleConnection Sample.Rental TestFramework.Coverage.Scripts
MSSQLDevice Sample.SendEmail TestFramework.StressTest
MySQLConnection Sample.Shipping

am I missing something?

Regards,

LuxSpes

OleDbClient

Hi,

The DAE.Connection assembly contains an OLEDBConnection class. This uses the Microsoft OleDb ADO.NET provider from the System.Data.OleDb namespace. In Microsoft.NET, this provider is part of the System.Data assembly, but I don't know whether or not the Mono project has implemented this provider. If the assembly (DAE.Connection) compiles, then they must have something implemented, so at this point I think it's probably the post build step discussed in the thread above. Does the development environment you are using handle the output directory being set to something like '..\'?

Regards,
Bryn Rhodes
Alphora