RE: Middle Tier (was Re: Table and Query Classes good or unnecessary?)
Hi Reg,
We build our own ERP software for managing our company processes. The performance of Omnis is not a problem for building the business logic. We currently do that and if we move this to the middle tier that is not an issue at all. The biggest performance issue is with adding a new layer in between the client application and the data in the database. Currently, without a middle tier the application for example does a select to get data. Omnis handles the connection to the server (all C++ code) by using the postgres libraries. So the request goes to the server, gets the data back (in as a binary), and it is put into an Omnis list and then you can continue running your Omnis logic. Which could be just display the list in a headed list box. This is very fast. We use background queries which fire off while typing a lot and you can get pretty big resultsets. When you need a middle tier you don’t directly go to the database, but you call the middle tier. That needs to fetch the data from the database (=fast), but then it needs to be sent back to the client. If you convert this to a JSON, send it as text to the client, convert this back to 1 or multiple lists this is quite a bit of overhead. When you have big result sets you will definitely notice this, but furthermore: if you build this communication in Omnis (as omnis does not provide this), this will add a noticeable delay. As I said in #4: this could probably solved by building an external component which handles this for you. The other place where the performance of Omnis is an issue is with manipulating big lists. This wasn’t a problem years ago, but the world has changed and you need to handle a lot more and more complex data. How lists work in Omnis the performance degrades exponentially, so this can be limiting in client side Omnis currently already. If you want to build up a middle tier for the future it would be nice if the platform is ready for that future as well.
So, I could currently build a communication protocol between Omnis server and Omnis client by building an external component, but then I still need a reliable, multi-threaded/multi-process platform.
Comments on your remarks:
1. It can be done, but it is really clunky, especially on windows. I don’t want to build an important part of my software on this, as this is will be a pain to maintain. Only Omnis can solve this elegantly.
2. There are a lot of small annoyances caused by this, but it doesn’t solve the processor core use. So now we ‘have’ to use remote tasks (which are created to keep state for Web/js client), but they only cause issues, not solving any. For example: if you want to use file classes for faking constants (as there still is no solution for this), you have to initialize them on every call (= adding useless overhead). You also need to be very careful when using class variables, because when you for example lazy-load a list variable from table class, you can end up with a half state (data is still there, but the instance which allows for adding functions to a list is killed), depending when that class variable gets instantiated. This can result in pretty hard to find bugs.
3. I think it is. At least on windows I have seen this in effect in our CAD-system. But reliability is definitely an important thing. Omnis is not the most crash-safe software I use and it can differ heavily between releases.
4. Yes, that is why I think Omnis could be viable as a middle tier, but they need to solve the communication and scalability part.
I think of the language first, framework second. And Bas doesn’t seem to be that different: He mentions node (which is basically Javascript for server, which mostly comes down to selecting that language as your server language), Ruby, which is a language, but you will probably use rails as a framework, and he mentions Python which just is a language and there are quite a few frameworks available for it.
I think Omnis could be a viable server platform for us, when they put in the effort. And I really hope they do, because nothing I know is as fast to develop business software/logic in. Omnis has been progressing very slowly for a decade, and this still remains true in my opinion. But they need to move to make it future proof again. I hope they can and will do!!
Regards,
Marten Verhoeven
—–Oorspronkelijk bericht—–
Van: omnisdev-en [mailto:omnisdev-en-bounces@lists.omnis-dev.com] Namens Reg Paling
Verzonden: donderdag 15 maart 2018 22:24
Aan: OmnisDev List – English <omnisdev-en@lists.omnis-dev.com>
Onderwerp: Re: Middle Tier (was Re: Table and Query Classes good or unnecessary?)
Hi Marten,
What’s your application? If you’re running a high-speed securities trading platform it’s going to have a very different sweet spot from running, say, a school where there is neither the same volume of data nor the urgency. Even on the trading platform only a small percentage of the code has to perform at the high level. So I think it makes sense to develop an architecture where the 80-90% of the code is in Omnis and the critical minor percentage of the code might need to be done with something else.
>> 1. Omnis doesn’t handle multi-threading / multi process very well.
It’s my impression that the workarounds for this shortcoming are not too onerous, i.e. the load-balancing setups may be a bit ugly but they work. Can those who have worked in this way, comment on this?
>> 2. Remote tasks … cached data/objects…
In my framework I have a handful of core task variables in the main task. My remote task superclass picks up a reference to these core task variables, and that way every remote task instance has access to cached data & the methods of these core instances.
>> 3. …If a call causes a crash, the complete service is taken down.
Yes, it is a big problem, and I have no idea whether it’s fixable except via the multiple Omnis instances mentioned in point 1.
>> 4. When adding a middle tier you add an extra layer…
This comes back to the 90%-10% principle. The 90% can be developed so efficiently in Omnis, and for this 90% middle tier logic in Omnis is fine.
My new application is for an educational organisation, so I don’t know to what degree I will ever need the high-performance middle tier we are talking about. Certainly my customers are going to be hoping that they will get occasional good media coverage that results in huge spikes of inquiries on their websites. I think that sort of aspiration plays a big part in the decision to use one platform or another.
It’s interesting that Bas put forward frameworks as the alternatives, whereas Marten you have put forward languages. If you go with a framework such as Rails, then that will tend pull your whole architecture into the Rails paradigm. If you go with a language, then you can keep your own vision but you have to piece together the whole middle-tier framework yourself.
I hope I am wrong but it seems to me that points 1 and 3 are always going to be a trade-off for Omnis. I hope Omnis Software comes up with one or two tightly-integrated solutions to say “Here’s how you best manage the high-performance portion of your application”.
Cheers,
Reg
On 15/3/18 8:39 pm, Marten Verhoeven wrote:
> Hi Reg,
>
> I think there are three problems currently with Omnis as a middle tier (maybe four):
>
> 1. Omnis doesn’t handle multi-threading / multi process very well. You have to install multiple instances to get multi process and add an intermediate layer in between to distribute the calls.
> 2. Remote tasks are currently supposed to handle calls, but they are instantiated on every request which makes it hard to use prepared or cached data/objects. So, if you instantiate a table class, you cannot keep the instance alive for a next request, which means you need to instantiate it every time, or only use the main thread (which basically means you don’t use remote tasks as intended). This means a lot of overhead which kills performance.
> 3. It is too unstable for a server-process. If a call causes a crash, the complete service is taken down. A crash of a call should only take down the execution of the current call, not the complete service.
> 4. When adding a middle tier you add an extra layer on top of the (very fast) database access. So this extra layer should be very fast, and Omnis is not very fast. Especially not in manipulating big lists. So core-improvements to get performance gains can be required (depending on what you do). Just making sure to get the data from the middle tier to the client quickly can also be achieved by building an external component just for that, so that might be enough.
>
>
> I have not investigated contenders heavily yet, but I would first look at:
>
> – C#: very capable language and I could also build some client side stuff in the same language (windows, iOS and android). It is a big ecosystem backed by and important for Microsoft, so it won’t go away.
> – Swift: Very capable and fast language and I could build some client
> side stuff in the same language (iOS only. Not being able to run on
> windows is a drawback). It will be remain a big ecosystem going into
> the future, as apple and IBM put a lot of weight behind it. Swift is
> the future for Apple, so that is pretty safe
> – Python Lots of libraries on the server-side, but I am not a fan of the language and the tools I currently know and it is not an option for client-side development.
> – Clojure – This is a very small contender. The only reason I am adding this is that I know common lisp pretty well and know how powerful and flexible the language is, but there probably are a lot of downsides to using it because it is a small ecosystem, not an option for client side development.
>
> I would also investigate whether I could use protocol-buffers for communication between server and client in order to get fast transfer between them. I’m not sure if it is dynamic enough though. There still is a lot to investigate for me.
>
> I do hope Omnis will be a viable option as well, because it is really flexible and very, very fast to develop in. And I can reuse a lot of code, so that is a big advantage as well.
>
> But client side Omnis requires a major update as well, in order to build more modern, dynamic user interfaces instead of static windows 2000 style interfaces.
>
> Both areas need focus and vision for me to have Omnis remain a long-term solution to build our software on.
>
> Regards,
>
> Marten Verhoeven
>
>
>
> —–Oorspronkelijk bericht—–
> Van: omnisdev-en [mailto:omnisdev-en-bounces@lists.omnis-dev.com]
> Namens Reg Paling
> Verzonden: dinsdag 13 maart 2018 0:30
> Aan:omnisdev-en@lists.omnis-dev.com >> OmnisDev List –
> English<omnisdev-en@lists.omnis-dev.com>
> Onderwerp: Re: Table and Query Classes good or unnecessary?
>
> Hi Bas & Marten,
>
> What enhancements do do want to see in Omnis, to better support the middle tier? What alternatives are you looking at, and what are their strengths etc?
>
> Cheers,
> Reg
>
> On 13/3/18 9:39 am, Bastiaan Olij wrote:
>> Hey Marten,
>>
>> On 12/3/18 8:55 pm, Marten Verhoeven wrote:
>>> If they just prioritize the JS client I will have to make different choices. If they focus on the fat cl
>>> ient to build your UI and on a better backend platform I will keep it all in Omnis.
>> Exactly the choice that is in my head too. The JS Client is nice but
>> its not a technology that currently interests me a lot. With where
>> Apple seems to be headed I much rather see a continuation of
>> enhancements in the fat client and as I’ve said many times before,
>> hopefully some day an iOS runtime which should be within our grasps
>> now that Studio 8 has seen the Mac OS X client rewritten.I believe it
>> is even inevitable as I think we’ll soon see iMacs or similar devices
>> running iOS. The JS Client I presume will remain a large part of
>> Omnis’ future seeing how successful it has been bringing in new
>> opportunities but I hope the company will see chances in the fat
>> client and improving the middle tier capabilities also helps the JS
>> client so that one is a given. The question for us is, will it be in
>> time? Right now the most likely future for us seems to be using Omnis
>> as a front end but another tool for the middle tier. The choice is
>> not made however 🙂
>>
>> That all said, I can only underline how enthusiastic I am for the
>> change of direction Omnis has shown. It’s definitely a matter of
>> giving them time for this to start showing fruits. It takes awhile
>> before a new direction gains momentum but I think we’ll be seeing
>> some exciting new things soon.
>>
>> Cheers,
>>
>> Bas
>>
>> _____________________________________________________________
>> Manage your list subscriptions athttp://lists.omnis-dev.com Start a
>> new message ->mailto:omnisdev-en@lists.omnis-dev.com
> _____________________________________________________________
> Manage your list subscriptions athttp://lists.omnis-dev.com Start a
> new message ->mailto:omnisdev-en@lists.omnis-dev.com
> _____________________________________________________________
> Manage your list subscriptions athttp://lists.omnis-dev.com Start a
> new message ->mailto:omnisdev-en@lists.omnis-dev.com
_____________________________________________________________
Manage your list subscriptions at lists.omnis-dev.com Start a new message -> mailto:omnisdev-en@lists.omnis-dev.com
_____________________________________________________________
Manage your list subscriptions at lists.omnis-dev.com
Start a new message -> mailto:omnisdev-en@lists.omnis-dev.com
