QCodeEdit

[kaos]

XML does induce some nasty escaping and a more complex parsing (even though this may not remain always true if you use QXmlStreamReader) but on the other hand it allows an easy and extremely powerful hierarchization and plain text can not compete with XML in that area...

Yes, i am a big fan of xml. But i found a custom format in that case much more managable.

How do you map a token with a given text format? Is that handled within the lexer which sneds the info along the token?

Each token gets a unique id when the lexer file gets parsed. With this unique id several infos about the token are saved (the name, the language, etc)

the "caching" is line based but instead of using an additional state variable I just check the context stack. Could you define what a "group" is in your syntax engine?

as far as i can see it's quite much the same as your context. it defines a certain "block" with a start and an end and additional rules for the text in the group.

Even with big files (about 1meg of C++ code) QCodeEdit remains responsive, unless you're nasty enough to place a multiline comment at the begining of such a big file and switch between commented and uncommented... But even this manipulation doesn't alter the responsiveness on "normal" files ( ~3-4k lines)

That's the part where the multithreaded model excels quite much. One other reason for me to go the threaded part were the emerging multithreaded/multicode cpus

Some people thing that threads make things faster but they actually make them way slower... The only advantage is responsiveness but what if responsiveness can be preserved without threading? The first version of Edyuk completion engine was threaded and it was SLOW (a couple of secs to display results). As soon as I dropped threading and replaced QListWidget by a QListView with a custom model it fell to a couple of msecs!!! It made me think a little more about threading and now I only use it when the speed loss is irrelevant in front of the responsiveness... In the case of highlighting I don't think threading is such a good idea but maybe some benchmarks could prove me wrong...

Well, making things threaded is of course more complicated then easy to get right. The lexer definitly doesn't need to be multi threaded, but i since the solution works really well, i saw no reason in changing that. I would not do the completer multithreaded tho, since the user usually expecting a response when he triggers auto completion. But for me personally any delay when typing is just not acceptable in an editor. And I didn't see any problem with lexing the stuff in an extra thread. I've ran my code on an p3 450 and it still worked rather smooth. I haven't benchmarked that entire thing tho, but iirc most of the time the lexer thread already was done with the line before the control went redrawing itself (on a single core/thread cpu)

[fullmetalcoder]

Each token gets a unique id when the lexer file gets parsed. With this unique id several infos about the token are saved (the name, the language, etc)

But then how do you handle parsing numbers for instance? It must be done with a more complex DFA than a keyword and allow for many more than one single token... I guess you meant that the id is associated with the DFA matching state or am I missing something?

[kaos]

But then how do you handle parsing numbers for instance? It must be done with a more complex DFA than a keyword and allow for many more than one single token... I guess you meant that the id is associated with the DFA matching state or am I missing something?

Ah, yeah.
Well, all the tokens get compiled into one big automaton per group where the final state carries information which token id to emit when the match completes at that final state.

[Usability]

I just noticed the license syntax selection is fooled by dots in the file name.
For example

example/example lib/qpanellayout.cpp

produces a correctly colourcoded file, while

cp lib/qpanellayout.cpp a.1.cpp
example/example a.1.cpp

Shows no colourcoding at all. I think the extension is the text after the final dot in a file name.

[fullmetalcoder]

I just noticed the license syntax selection is fooled by dots in the file name.
For example

example/example lib/qpanellayout.cpp

produces a correctly colourcoded file, while

cp lib/qpanellayout.cpp a.1.cpp
example/example a.1.cpp

Shows no colourcoding at all. I think the extension is the text after the final dot in a file name.

The issue must come from a use of QFileInfo::completeSuffix()... I'll correct it ASAP.

[Usability]

How do you feel about making ctrl-f always take you to the find box?
Currently you have to first check if the find box is open, because if it is, you need to key two ctrl-f to get there. (The first one just closes the box, the second opens it again and takes you in).

I think it would be much easier this way. Conventionally you use escape to leave a dialog, not a repeat of the key that opened it.

[fullmetalcoder]

How do you feel about making ctrl-f always take you to the find box?

I'll have that sorted out ASAP.

BTW, the suffix issue has been fixed in case you didn't notice.

[travlr]

Hi fmc,
A couple of issues/observations for edyuk.

• First, I don't understand... Is not the trunk in SVN repo not the most recent code??? Huh? This quite unorthodox, isn't it. I've never seen this before. Quote from the website:

  ...and, as many people dislike (or don't even know) SVN, I ended up releasing a new package to avoid confusions with the very old one left on Sourceforg donwloads servers...
  
  You can get it in the downloads section and enjoy all the improvements.

• The ./build script reports a syntax error

  oxbo% ./build -h
  ./build: 60: Syntax error: Bad substitution

  oxbo% ./build -h
  ./build: 60: Syntax error: Bad substitution

  To copy to clipboard, switch view to plain text mode 
• Using qt-4.4, a Make error regarding QAtomic and QBasicAtomic (4.4 has QAtomicInt and QAtomicPointer, Before 4.4 there is no "QAtomic*" at all according to the class refs)

  In file included from ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/qeditor.cpp:19:
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/document/qdocument_p.h:32:19: error: QAtomic: No such file or directory
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/document/qdocument_p.h:215: error: ‘QBasicAtomic’ does not name a type
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/document/qdocument_p.h: In member function ‘void QDocumentCursorHandle::ref()’:
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/document/qdocument_p.h:202: error: ‘m_ref’ was not declared in this scope
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/document/qdocument_p.h: In member function ‘void QDocumentCursorHandle::deref()’:
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/document/qdocument_p.h:203: error: ‘m_ref’ was not declared in this scope
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/document/qdocument_p.h:203: error: ‘m_ref’ was not declared in this scope
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/qeditor.cpp: In member function ‘virtual bool QEditor::moveKeyEvent(QDocumentCursor&, QKeyEvent*, bool*)’:
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/qeditor.cpp:1973: warning: unused variable ‘prevcol’
  make[2]: *** [.build/4.4.0-tp1-unix/obj/release/qeditor.o] Error 1
  make[2]: Leaving directory `/d/bld/edyuk/src/lib'
  make[1]: *** [release] Error 2
  make[1]: Leaving directory `/d/bld/edyuk/src/lib'
  make: *** [sub-src-lib-lib-pro-make_default] Error 2
  makeobj[0]: Leaving directory `/d/bld/edyuk'

  In file included from ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/qeditor.cpp:19:
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/document/qdocument_p.h:32:19: error: QAtomic: No such file or directory
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/document/qdocument_p.h:215: error: ‘QBasicAtomic’ does not name a type
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/document/qdocument_p.h: In member function ‘void QDocumentCursorHandle::ref()’:
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/document/qdocument_p.h:202: error: ‘m_ref’ was not declared in this scope
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/document/qdocument_p.h: In member function ‘void QDocumentCursorHandle::deref()’:
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/document/qdocument_p.h:203: error: ‘m_ref’ was not declared in this scope
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/document/qdocument_p.h:203: error: ‘m_ref’ was not declared in this scope
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/qeditor.cpp: In member function ‘virtual bool QEditor::moveKeyEvent(QDocumentCursor&, QKeyEvent*, bool*)’:
  ../../../../src/edyuk-1.0.0-pre1/3rdparty/qcodeedit2/lib/qeditor.cpp:1973: warning: unused variable ‘prevcol’
  make[2]: *** [.build/4.4.0-tp1-unix/obj/release/qeditor.o] Error 1
  make[2]: Leaving directory `/d/bld/edyuk/src/lib'
  make[1]: *** [release] Error 2
  make[1]: Leaving directory `/d/bld/edyuk/src/lib'
  make: *** [sub-src-lib-lib-pro-make_default] Error 2
  makeobj[0]: Leaving directory `/d/bld/edyuk'

  To copy to clipboard, switch view to plain text mode 

This was just a first look, before investigating reasons or fixes.

Thank you,
-travlr

[Usability]

Double click in the editor got broken recently. With a slightly older QCodeEdit, double clicking selects a word. In the latest svn, double click does nothing.

[fullmetalcoder]

@travlr :

• There must be a misunderstanding somehow... SVN repo always hold the most recent code AND from time to time I package the content of the repo and put in on Sf.net dowload servers so that more people can play with it (as some are reluctant to use SVN...)
• Would you be running under BSD? Or some exotic UNIX system? I must admit I'm not an expert of clean shell scripting and I'd be able if someone could help me to improve the portability of the build script...
• Do you know what I need to do in order to make it compatible with Qt 4.4 then? Looks like it's just a matter of include files. Or did the QBasicAtomic class disappear completely in the utter nothingness?
• Some of your comments concern Edyuk ad not QCodeEdit so it may be better to post in the appropriate thread.

@Usability : I fixed the double click regression.

[Usability]

Double click working again, thanks!

[fullmetalcoder]

How do you feel about making ctrl-f always take you to the find box?

Done.

A couple other changes have happened lately. Among which the addition of a line change panel which indicates the modified lines since last save.

[fullmetalcoder]

A couple more good news :

• caduel has just fixed the code to work with Qt 4.4 (well actually it broke Qt 4.3 backward compat but I fixed that by now so everything is fine regarding this issue
• I achieved very significant speed improvements when editing veryl arge files (especially near the end where it used to lag...)
• Fixed cursor auto update mechanism in combination with cursor mirrors (typing enter when using cursor mirrors used to wreck cursor positions in a somewhat weird way.)

[elcuco]

What was the problem with Qt 4.4? it should compile perfectly, since 4.4 is source compatible with 4.3.

[fullmetalcoder]

What was the problem with Qt 4.4? it should compile perfectly, since 4.4 is source compatible with 4.3.

It's described in a previous post... Contrary to what one could expect there's been some, at least one, API changes which happened in the QBasicAtomic class (got hidden since the introduction of QAtomicInt AFAIK). This class is a low level atomic counter used in reference counting (in both text lines and cursors).

[jpn]

They weren't part of the public API, were they?

[elcuco]

It's described in a previous post... Contrary to what one could expect there's been some, at least one, API changes which happened in the QBasicAtomic class (got hidden since the introduction of QAtomicInt AFAIK). This class is a low level atomic counter used in reference counting (in both text lines and cursors).

BTW, I am getting compilation problems on 4.3. What version do you support ...? 4.4? 4.3?

[fullmetalcoder]

BTW, I am getting compilation problems on 4.3. What version do you support ...? 4.4? 4.3?

I'd like to support both but I can't test Qt 4.4
As for your compile troubles could provide me the following informations :

• OS
• Qt version
• compiler
• QCodeEdit version (i.e package or SVN rev)
• full compile log (do a make distclean first if you are using a SVN copy)

They weren't part of the public API, were they?

The QAtomic and QBasicAtomic class are not documented but they are accessible (i.e. not located somewhere in "private" headers) and as they are widely used throughout Qt and the only way to achieve atomic ref counting I assumed they would not be a source of troubles and used them...