QCodeEdit

[fullmetalcoder]

hi, I've had planned quite a while ago to rewrite the text editing module of Edyuk and, as I was bored with writing Edyuk docs last week, I started thinking about that.

QCodeEdit is meant to be a framework providing :

• a powerful and flexible widget mixing a QTextEdit and various panels
• an easily extendable framework for "language definitions" : i.e. each programming language is associated with one or more file extensions and gets its own parenthesis matcher, text indenter and syntax highlighter
• an wide set of powerful generic components that interfaces between XML definitions and "language framework"

The archive I'm releasing is just a first draft and currently features only the core of the language framework and the generic highlighting engine.

For simplicity's sake it's gonna be possible to attach the language framework to a QTextEdit or to QCodeEdit indifferently.

The beta is bundled with a minimal example and an XML definition for C++ (with Doxygen stuff)

Have fun and feed me back.

[elcuco]

Anyone tred this code...?

Anyway, I tested your code. In order to make it run I needed to apply this patch:

diff -ru qcodeedit-0.1.0-beta-orig/example/example.pro qcodeedit-0.1.0-beta/example/example.pro
--- qcodeedit-0.1.0-beta-orig/example/example.pro       2006-07-28 16:37:06.000000000 +0300
+++ qcodeedit-0.1.0-beta/example/example.pro    2006-08-01 23:03:25.000000000 +0300
@@ -4,8 +4,10 @@
 
 TEMPLATE = app
 TARGET = example
-
+INCLUDEPATH +=  ../include ../src ../src/editor ../src/language ../src/generic
+DEFINES += _QCODE_EDIT_GENERIC_
 CONFIG += release qcodeedit
+LIBS += -L../src/ -lqcodeedit

diff -ru qcodeedit-0.1.0-beta-orig/example/example.pro qcodeedit-0.1.0-beta/example/example.pro
--- qcodeedit-0.1.0-beta-orig/example/example.pro       2006-07-28 16:37:06.000000000 +0300
+++ qcodeedit-0.1.0-beta/example/example.pro    2006-08-01 23:03:25.000000000 +0300
@@ -4,8 +4,10 @@

 TEMPLATE = app
 TARGET = example
-
+INCLUDEPATH +=  ../include ../src ../src/editor ../src/language ../src/generic
+DEFINES += _QCODE_EDIT_GENERIC_
 CONFIG += release qcodeedit
+LIBS += -L../src/ -lqcodeedit

To copy to clipboard, switch view to plain text mode 

I also needed to copy the *.so file into the example dir. When you type text the application does work, but when pasting large portions of text (qwidget.cpp was my test case) or load it via the command line the text does not get painted.

About the code itself:
The idea of having different contexts in the language definitions if cool. This means that different color can be applied to the same word depending of it's location on the text.

I am not sure that putting the colors definitions inside the syntax declaration is a good idea: you are not able to modify the colors of your syntax. For example, people which have a pink desktop will have problems with your code, since the text editor will always be with white background.

I am not sure that re-inventing the wheel and defining a new syntax declaration is a good idea. If you choose another project language definitions are will gain a lot of syntaxes for free, and you now only have to code an engine to display them. For example:

Sharp develop also use XML for describing the syntaxes:
http://www.icsharpcode.net/OpenSource/SD/

GtkSourceView also uses XML (see also my minimal implementation for this definitions)
http://gtksourceview.sourceforge.net/

And also notepad++ (win32 application):
http://notepad-plus.sourceforge.net/uk/site.htm

Another suggestion can be using plain "ini" files instead XML. This will make the construction on new syntax definition files much easier.

[Matt Smith]

The convention in Qt is that only Trolltech issues classes with names like QClassName. Third-party developers are expected to put initials after the Q to identify it as theirs, as in QfmcCodeEdit. You can find the naming guidelines here. You are supposed to register the initials with Trolltech, but I have registered my own (ij) and have not heard from them and it has not appeared on their site.

[fullmetalcoder]

Anyone tred this code...?

Anyway, I tested your code. In order to make it run I needed to apply this patch:

diff -ru qcodeedit-0.1.0-beta-orig/example/example.pro qcodeedit-0.1.0-beta/example/example.pro
--- qcodeedit-0.1.0-beta-orig/example/example.pro       2006-07-28 16:37:06.000000000 +0300
+++ qcodeedit-0.1.0-beta/example/example.pro    2006-08-01 23:03:25.000000000 +0300
@@ -4,8 +4,10 @@
 
 TEMPLATE = app
 TARGET = example
-
+INCLUDEPATH +=  ../include ../src ../src/editor ../src/language ../src/generic
+DEFINES += _QCODE_EDIT_GENERIC_
 CONFIG += release qcodeedit
+LIBS += -L../src/ -lqcodeedit

diff -ru qcodeedit-0.1.0-beta-orig/example/example.pro qcodeedit-0.1.0-beta/example/example.pro
--- qcodeedit-0.1.0-beta-orig/example/example.pro       2006-07-28 16:37:06.000000000 +0300
+++ qcodeedit-0.1.0-beta/example/example.pro    2006-08-01 23:03:25.000000000 +0300
@@ -4,8 +4,10 @@

 TEMPLATE = app
 TARGET = example
-
+INCLUDEPATH +=  ../include ../src ../src/editor ../src/language ../src/generic
+DEFINES += _QCODE_EDIT_GENERIC_
 CONFIG += release qcodeedit
+LIBS += -L../src/ -lqcodeedit

To copy to clipboard, switch view to plain text mode 

I also needed to copy the *.so file into the example dir. When you type text the application does work, but when pasting large portions of text (qwidget.cpp was my test case) or load it via the command line the text does not get painted.

About the code itself:
The idea of having different contexts in the language definitions if cool. This means that different color can be applied to the same word depending of it's location on the text.

I am not sure that putting the colors definitions inside the syntax declaration is a good idea: you are not able to modify the colors of your syntax. For example, people which have a pink desktop will have problems with your code, since the text editor will always be with white background.

I am not sure that re-inventing the wheel and defining a new syntax declaration is a good idea. If you choose another project language definitions are will gain a lot of syntaxes for free, and you now only have to code an engine to display them. For example:

Sharp develop also use XML for describing the syntaxes:
http://www.icsharpcode.net/OpenSource/SD/

GtkSourceView also uses XML (see also my minimal implementation for this definitions)
http://gtksourceview.sourceforge.net/

And also notepad++ (win32 application):
http://notepad-plus.sourceforge.net/uk/site.htm

Another suggestion can be using plain "ini" files instead XML. This will make the construction on new syntax definition files much easier.

I'm pleased to have some feedback but I must inform you that the code I posted here is quite out-of-date... I've continued developping it for use with Edyuk and you can always find the latest version on the SVN trunk or (a little older one) on the latest surce package.

The syntax definition in itself still needs some work. Honestly I didn't pretend to have crafted the ultimate file format here. I just thought about one that would be human readable and has some convinience. I'm not against using other ones but as you might have noticed I'm using some rather special mechanism like the href stuff and the context aware definitions (or nested definitions). Besides if you want to support another generic definition format all you need is to reimplement the some interfaces for highlighting, paren matching, ... and also create and add proper definitions to the factory.

About the colors : I didn't had much time to think about that but you're probably right... I should place them in a separate file. I'll give it a try in a next version.

Using INI files should allow faster and a little easier syntax creation but it would also make them look ugly when using nested/context-aware highlighting definitions...

The convention in Qt is that only Trolltech issues classes with names like QClassName. Third-party developers are expected to put initials after the Q to identify it as theirs, as in QfmcCodeEdit. You can find the naming guidelines here. You are supposed to register the initials with Trolltech, but I have registered my own (ij) and have not heard from them and it has not appeared on their site.

I had never heard of that and, IMHO, it's rubbish... Long prefixes make the code longer to type and sometimes harder to understand. Moreover, everytime I use the Q prefix it's to state a self-sufficient component (apart from Qt dependencies of course ) that is meant to become someday a library well-integrated into Qt... Ultimately, if the Trolls asked me to change that I'd rather put a namespace than changing all class names...

[No-Nonsense]

I'm pleased to have some feedback but I must inform you that the code I posted here is quite out-of-date... I've continued developping it for use with Edyuk and you can always find the latest version on the SVN trunk [...]

I would like to try out the latest version of QCodeEdit as I am programming a source editor for some custom language.
I started with a QTextEdit, added a QSyntaxHighlighter and wanted to have line-numbers and later syntax completion and icons on the line-number-gutter. I just started a own editor based on QTextEdit and using various Qt editors as a base how to implement a gutter etc.

Sadly I cannot compile your source as the *.pro file is missing in ./src/lib/qcodeedit/ or do you have a separate SVN repository for QCodeEdit?

The syntax definition in itself still needs some work. Honestly I didn't pretend to have crafted the ultimate file format here. I just thought about one that would be human readable and has some convinience. I'm not against using other ones but as you might have noticed I'm using some rather special mechanism like the href stuff and the context aware definitions (or nested definitions).

You could look at the Katepart language definitions. Personally I dislike some of their syntax (some XML elements start with a lowercase char, some with uppercase...; some of the rules are overdefined, e.g. matching a decimal/hex/... can be done with a regexp instead too). But what you could add support for a context stack (or isn't this needed by your definitions?) and standard colors.

About the colors : I didn't had much time to think about that but you're probably right... I should place them in a separate file. I'll give it a try in a next version.

I would prefer the way Katepart solves this: They define some standard colors (keyword, comment, decimal, hex, float, ...). The user (programmer) can then set the colors and styles for these. In the syntax definition file you would allow to set a color or to use one of these placeholders. Then QCodeEdit would init the associated colors with some defaults the programmer can overwrite at any point with other values.
You could also extend this to letting the syntax definition define the placeholder-names and offer a way for the programmer to get a list of names and build a config dialog for the user to set the colors.

-Jens

[travlr]

-Jens

Jens, are you the same no-nonsense from Elite Trader??

[No-Nonsense]

Jens, are you the same no-nonsense from Elite Trader??

No, don't think so. What is Elite Trader?

-Jens

[travlr]

No, don't think so. What is Elite Trader?

-Jens

Oh.. Ok. I guess not.
ET is a forum where financial market traders gather.

-travlr

[fullmetalcoder]

I would like to try out the latest version of QCodeEdit as I am programming a source editor for some custom language.
I started with a QTextEdit, added a QSyntaxHighlighter and wanted to have line-numbers and later syntax completion and icons on the line-number-gutter. I just started a own editor based on QTextEdit and using various Qt editors as a base how to implement a gutter etc.

Sadly I cannot compile your source as the *.pro file is missing in ./src/lib/qcodeedit/ or do you have a separate SVN repository for QCodeEdit?

QCodeEdit is a componnent and when released as is it contains a pro file to compile it as a static or shared lib. However I thought it would be absolutely useless to keep this pro file in Edyuk repositories and packages. Instead I split the pro files into : a wrapper .pro that's quite simple and distributed only in QCodeEdit packages and a .pri file (PRoject Include) which has to be included in your project via the include() function...

If you want to start playing with QCodeEdit I really recommend to use the iclude() function at first because it will spare you the pain of compiling an lib and then linking your app to it... However if you need a sample pro file to build and install QCodeEdit as a regular lib on your system : just ask me, I'll craft you one in a few minutes...

You could look at the Katepart language definitions. Personally I dislike some of their syntax (some XML elements start with a lowercase char, some with uppercase...; some of the rules are overdefined, e.g. matching a decimal/hex/... can be done with a regexp instead too). But what you could add support for a context stack (or isn't this needed by your definitions?) and standard colors.

I don't quite understand your last sentence...

I would prefer the way Katepart solves this: They define some standard colors (keyword, comment, decimal, hex, float, ...). The user (programmer) can then set the colors and styles for these. In the syntax definition file you would allow to set a color or to use one of these placeholders. Then QCodeEdit would init the associated colors with some defaults the programmer can overwrite at any point with other values.
You could also extend this to letting the syntax definition define the placeholder-names and offer a way for the programmer to get a list of names and build a config dialog for the user to set the colors.

Yep! I got to do it that way but unfortunately I'm ATM kept rather busy by the developpment of Edyuk and such improvements will have to wait a bit.

[No-Nonsense]

QCodeEdit is a componnent and when released as is it contains a pro file to compile it as a static or shared lib.

This may be OT here, but with my simple extended QTextEdit (only line-number-gutter) I had problems to get it working as a static library. I could get it to work as a shared library (plugin for Qt Designer) or by simply adding all files to the source an header variable in the *.pro and using "promote to custom widget".

I already searched the forums here, but did not find any usefull hints. I only changed the plugin pro file and added the static keyword there. Do I have to change anything else to get this to work? Maybe I can send you my test sources?

Another problem I had was acessing member objects of my custom TextEdit (i.e. the Gutter). I always had to include the header (ok) but also add the *.h and *.cpp to the project sources & headers when using as a plugin. Is there a way to only use the library and the *.h files? Because if not the plugin makes no sense to me as I could use it directly by including all files.

If you want to start playing with QCodeEdit I really recommend to use the iclude() function at first because it will spare you the pain of compiling an lib and then linking your app to it... However if you need a sample pro file to build and install QCodeEdit as a regular lib on your system : just ask me, I'll craft you one in a few minutes...

Could you provide me a samle project please?

I don't quite understand your last sentence...

In the Kate syntax definition you define some sort of a statemachine. There is a start context and rules that can stay or change the context. Then you are able to push and pop the current context while changing it. An example is given here: http://docs.kde.org/stable/en/kdebas...ml-format.html

Thanks in advance,
-Jens

[fullmetalcoder]

Could you provide me a samle project please?

Sure! But I need an e-mail address...

In the Kate syntax definition you define some sort of a statemachine. There is a start context and rules that can stay or change the context. Then you are able to push and pop the current context while changing it. An example is given here: http://docs.kde.org/stable/en/kdebas...ml-format.html

The Kate syntax engine sounds nice and as far as I've played with it, decently fast... However the problem I have is simple : time... Unless their implementation is small enough and can easily be ported to Qt4/QCodeEdit I won't try to use their definition format because I'm quite busy ATM...

[fullmetalcoder]

Here comes the fun! I finally took a few minutes to package the last source of QcodeEdit and released them as QCodeEdit 1.0 on Edyuk's Sf.net donwload server.

Hoping this will help those who need a ready-made editing framework and motivate those asking for a new syntax engine...

[magland]

Looking forward to trying it out...

Had some trouble compiling. The readme.txt is confusing, because there are three .pro files in the main directory of distribution.

... so I tried
>> qmake qcodeedit.pro
>> make

But this sent me into a recursive situation ??
qmake -win32 -o Makefile qcodeedit.pro
qmake -win32 -o Makefile qcodeedit.pro
qmake -win32 -o Makefile qcodeedit.pro
qmake -win32 -o Makefile qcodeedit.pro
qmake -win32 -o Makefile qcodeedit.pro
qmake -win32 -o Makefile qcodeedit.pro
....

I'm using Windows with MinGW
Help?

[fullmetalcoder]

Looking forward to trying it out...
I'm using Windows with MinGW
Help?

Apologies... I made a shell script but no batch equivalent... Anyway it shouldn't be to hard... Just copy makeall.sh as makeall.bat, open it as text and remove the first line (you may also need to replace every occurence of "make" by "mingw32-make") and everything should be fine.

[magland]

Thanks, the library seemed to compile, but now I'm having trouble compiling example.pro. I get the following error. Any advice?

tmp\obj\main.o(.text+0x171): In function `Z5qMainiPPc':
C:/cstuff/qcodeedit-1.0/main.cpp:26: undefined reference to `QPlainTextEditCreat
or::id()'
tmp\obj\main.o(.text+0x192):C:/cstuff/qcodeedit-1.0/main.cpp:26: undefined refer
ence to `QFoldPanelCreator::id()'
tmp\obj\main.o(.text+0x1b3):C:/cstuff/qcodeedit-1.0/main.cpp:26: undefined refer
ence to `QLineNumberPanelCreator::id()'
tmp\obj\main.o(.text+0x1d4):C:/cstuff/qcodeedit-1.0/main.cpp:26: undefined refer
ence to `QLineMarkPanelCreator::id()'
collect2: ld returned 1 exit status
mingw32-make[1]: *** [release\example.exe] Error 1
mingw32-make[1]: Leaving directory `C:/cstuff/qcodeedit-1.0'
mingw32-make: *** [release] Error 2

[fullmetalcoder]

Whoops... I almost forgot how stupid Window$ is since I have moved to Linux...

The problem is that some classes need explicit export so the linking to a shared lib fails. You can either fall back to the standalone program (standalone.pro) or set the library to compile as static (the first option is the best because you are likely to face the opposite problem with the second one... i.e. explicit shared import from a static lib...)

A third option would be to patch files by hand :

in src/defs.h :
replace : class c##Creator : public Q##t##Creator \
by : class Q_API c##Creator : public Q##t##Creator \

[mikhailt]

Thaks a lot for your library! The idea is very good, you made a very important and common used component

Libary
I build QCodeEdit on Win with Qt 4.2.2, but project need some small changes to be compiled

1. qmargin.h should be included in qeditor.h
#include "qmargin.h"

2. qlanguagefactory.h line 61
void setLanguage(QCodeEdit *e, QLanguageDefinition *d, QCompletionEngine *e);
here was the redefinition of variable.'e'
btw, variable names like a,b,c,d is a bad style

Eample project.
1. Makefiles for example, generated by qmake, should be edited a bit.
To compile axample I added a full path to qcodeedit.lib to Makefiles.

2. Then Q_ID in main.cpp statement
def << Q_ID(QLineMarkPanel)
<< Q_ID(QLineNumberPanel)
<< Q_ID(QFoldPanel)
<< Q_ID(QPlainTextEdit);
does not work properly and generate 4 unresolved externals for functions like QLineMarkPanelCreator, QFoldPanelCreator and so on.
Btw macros are evil, a bad, unclear programming way.

3. To resolve externals file #include "qeditorfactory.h" should be included i n main.cpp

4. But file qeditorfactory.h sontains to links to
#include "edyuk.h"
#include "qsettingsclient.h"
These are not present in the project at all. This comes us to lots of errors.

Thus example can not be compiled unfortunately and it remains unclear what this QCodeEdit does.
Good library but useless, because nobody, except author, knows how to use it

[elcuco]

I have tested this library today. Here are my toughts:

1. The instructions for building are wrong. The instructions for building the project says to run qmake; make. The project is buildable using the makeall.sh scripts. Note that you also point to the documentation, but it's not shiped, and you do not explain how to create it.
2. No Doxyfile available. There is not Doxyfile (or something similar) on the project, which makes it impossible to create the documentation. I have to build a Doxyfile myself. I can also note that the documentatino tries to be very detailed but it fails, and many things (for example the XLD specifications) are just not done.
3. The build system is not smart. Using a subdirs project and keeping a single Makefile per directory, will be smarter. Porting the demo/librabry to other build systems might be also a good idea.
4. Some files on the tarball are +X. For example LGPL.txt. Nothing important.
5. The some subdirectories contain a .svn directory. Which means the tarball is not really clean, see (4) as well. Nothing important.
6. The language defintions, are not done in a smart way. For example, there is not separation of language definitions and dispplay: the colors and fonts are embbeded into the language definitions, what if I don't want keywords to be hightlighted? Also, the hightlight/keyword/words list is definetly a bad idea. The word list is saved as a single string instead of a list of nodes, which means that the library needs to load that node and then re-parse it. Also, the doxygen implementation is not optimal, as it only takes into account words, and is not aware of contents. While we are talking about doxygen, the c++ and python share the same doxygen definitions, which means that I hoped that both syntaxes would include the doxygen domain. Instead, you have 3 implementations of doxygen: in cpp.xld, python.xld and dox.xld.
7. Using popups for search, is not a good idea. Modern applications should use something similar to vim/Firefox search. And besides, the search was not triggere on my box by the ctfl+f shortcut. Actually, even after I choosed it from the pop-up menu, it did not find the word "include" in a cpp file.
8. When I run the application, is uses 100% cpu, and 50% for X, even at idle states. The library needs a little more fine tunning, otherwise it is just not usable.
9. The license is not clear. The text says "You may use, distribute and copy QCodeEdit binaries under the terms of GNU Library General Public License (LGPL) version 2", and then "QCodeEdit sources are considered as part of Edyuk and thus GNU General Public License (GPL) version 2". Note also that LGPL and GPL talk generally about source code and not binaries. What are my rights regarding the library?
10. The code is not FreeSoftware. At the top of GPL.txt you add new restrictions to the GPL, which is is not permitted by the GPL. Look at point (6) of the GPL.
11. I could not make the completion work.

[fullmetalcoder]

@ mikhailt :

I don't like macros myself but I didn't find any other way to achieve editor modularization... Besides if the example did not build it is most likely due not using the build script (but I should have made this clearer and provided a window equivalent... ). FYI information the QEditorFactory is a convinience class that's not fully part of the QCodeEdit library because of its dependency on Edyuk. However I left it here because it can always be ported to any other app just by replacing (or removing) the exporting macros and implement another settings handling. I might take time to remove these dependencies later on while keeping all QEditorFactory's functionalities but I've not much time now...

@ elcuco :

1,2,3,4,5 : Whoops... You're right once again... Got to correct this.
6 : I've explained already why definitions are what they are currently. I probably won't have time to change that before a long time but after having a look at the generic components that are bundled (and handle the XLD files) one, especially you , should be able to build proper interfaces to support Kate (or any other) syntax definitions.
7 : that's your opinion but I assume I should just provide default search/replace widget of both type...
8 : Well, whe I use QCodeEdit, even from Edyuk, I never face such a situation except maybe on loading of big files but at least the UI doesn't freeze...
9 : Not clear? Well I think it would be... There are basically two cases :

• One embeds QCodeEdit into an app or modify it and thus MUST release its work under GPL
• One just links to an UNMODIFIED version of QCodeEdit and undergo no restrictions apart from these stated by the LGPL (which are not very tough, are they?)

10 : Someone told me this already but I forgot to modify the files...
11 : the completion is not build-in... There is just a ready framework that only needs a proper QCompletionEngine to be implemented and registered by the app. It seemed to me that it would be impossible to implement an efficient completion in a generic way so I only kept the raw C++ interface.

I will try to issue a fixed version of QCodeEdit ASAP taking all your remarks into account but please remember that I'm rather busy ATM.

[fullmetalcoder]

Good news from the front line!

After a cleanup of Edyuk's SVN trunk the latest sources of QCodeEdit are now available directly from SVN with bundled examples and proper project files :

$ svn checkout http://edyuk.svn.sourceforge.net/svn...arty/qcodeedit qcodeedit

I'm working on a new spec for generic highlighting definitions which should be much more flexible and fix most (if not all) issues reported here.