6:56 a.m.
Greetings,
I implemented a simple common option (-K) for the client tools that if
enabled, sets the file descriptor used to output errors to stdout instead
of the default (and currently used) stderr.
This is done because when running a batch of commands with the output sent
to a file, logs collected are not shown chronologically because the output
is split between stdout and stderr. This occurs because both stdout and
stderr do not have the same buffer behavior and unfortunately, there is no
means within any shell to get them to behave in concert.
Here is an example of the output of ldapmodify saved in a file using the
current code:
ldapmodify: Type or value exists (20)
ldapmodify: Type or value exists (20)
ldapmodify: Type or value exists (20)
modifying entry "uid=User01,ou=MyOrganization,o=example.com "
modifying entry "uid=User03,ou=MyOrganization,o=example.com "
modifying entry "uid=User04,ou=MyOrganization,o=example.com "
modifying entry "uid=User03,ou=MyOrganization,o=example.com "
modifying entry "uid=User02,ou=MyOrganization,o=example.com "
modifying entry "uid=User05,ou=MyOrganization,o=example.com "
modifying entry "uid=User21,ou=MyOrganization,o=example.com "
modifying entry "uid=User22,ou=MyOrganization,o=example.com "
Now when instructing ldapmodify to redirect errors to stdout, the
following output shows up.
modifying entry "uid=User01,ou=MyOrganization,o=example.com "
modifying entry "uid=User03,ou=MyOrganization,o=example.com "
ldapmodify: Type or value exists (20)
modifying entry "uid=User04,ou=MyOrganization,o=example.com "
ldapmodify: Type or value exists (20)
modifying entry "uid=User03,ou=MyOrganization,o=example.com "
modifying entry "uid=User02,ou=MyOrganization,o=example.com "
ldapmodify: Type or value exists (20)
modifying entry "uid=User05,ou=MyOrganization,o=example.com "
modifying entry "uid=User21,ou=MyOrganization,o=example.com "
modifying entry "uid=User22,ou=MyOrganization,o=example.com "
As you can see, in the first case it is impossible to tell which of the
multiple operations failed. With my little feature, now it is possible.
Many persons on the web complain about this and I believe that my feature
will be well received. Default for errors still remain stderr, so normal
usage will remain undisturbed.
The change to the code is very simple. I change common.c and common.h,
added the -K options and replaced stderr with errorfp (default to stderr)
in many of the tools' source code. Very low risk of breaking anything. I
have been using ldapsearch and ldapmodify a lot with my option. Why -K?
Because it is one of the rare letters still available in the long list of
options.
I would like to submit my changes for review and hopefully, if accepted,
that it becomes part of the trunk. However, I do not know how to proceed.
Though I am familiar with other configuration management tools, I am not
familiar with git, thus if any operation with this tools is required,
please provide me examples of commands I should do.
Best regards,
Hans Deragon
http://www.deragon.biz
4:33 a.m.
Hans Deragon wrote:
I implemented a simple common option (-K) for the client tools that
if
enabled, sets the file descriptor used to output errors to stdout
instead
of the default (and currently used) stderr.
This is done because when running a batch of commands with the output
sent
to a file, logs collected are not shown chronologically because the
output
is split between stdout and stderr. (...)
I suggest instead a -B option to make stdout unbuffered or
linebuffered.
Hallvard
6:21 a.m.
Hallvard Breien Furuseth wrote:
Hans Deragon wrote:
> I implemented a simple common option (-K) for the client tools that
> if
> enabled, sets the file descriptor used to output errors to stdout
> instead
> of the default (and currently used) stderr.
That would be stupid. Error messages belong on stderr, that's why it exists.
> This is done because when running a batch of commands with the
output
> sent
> to a file, logs collected are not shown chronologically because the
> output
> is split between stdout and stderr. (...)
There are two obvious fixes:
1) don't send batches of commands that ignore errors, so that the
processing immediately stops at the failed entry.
2) change the error message text to include the input line number where it
failed.
Also a possibility:
don't send running status messages to stdout, send them only to stderr.
Technically the only commandline tool that normally produces "output" is
ldapsearch. Everything else is "diagnostic" and belongs on stderr anyway.
I suggest instead a -B option to make stdout unbuffered or
linebuffered.
stdout is already linebuffered by default when directed to a tty.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
12:50 p.m.
Greetings M. Chu,
The most practical solution is to send running status messages to one
stream, as you suggest, though I would send it to stdout, not stderr. In
my view, stderr is for errors and diagnostics in the sense of debugging,
not for normal operation. A tool that reports the status of its
operations should send its output to stdout. Connection errors, timeouts
and such abnormal errors should be reported to stderr. Duplicate values,
existing values, entry not found and such are normal operational messages
and should be sent to stdout.
I made my changes because they were the simplest, fasted solutions which
do not modify the current tools default behavior. I also believe that in
the context of logging, using one single stream to record all events in
chronological order is very practical. Since we disagree on this matter
and you are chef architect of OpenLDAP, you decision stands and I respect
that. We will leave the tools as is.
Best regards,
Hans Deragon
Hallvard Breien Furuseth wrote:
> Hans Deragon wrote:
>> I implemented a simple common option (-K) for the client tools that
>> if
>> enabled, sets the file descriptor used to output errors to stdout
>> instead
>> of the default (and currently used) stderr.
That would be stupid. Error messages belong on stderr, that's why it
exists.
>> This is done because when running a batch of commands with the output
>> sent
>> to a file, logs collected are not shown chronologically because the
>> output
>> is split between stdout and stderr. (...)
There are two obvious fixes:
1) don't send batches of commands that ignore errors, so that the
processing immediately stops at the failed entry.
2) change the error message text to include the input line number where
it
failed.
Also a possibility:
don't send running status messages to stdout, send them only to stderr.
Technically the only commandline tool that normally produces "output" is
ldapsearch. Everything else is "diagnostic" and belongs on stderr anyway.
> I suggest instead a -B option to make stdout unbuffered or
> linebuffered.
stdout is already linebuffered by default when directed to a tty.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
9:10 a.m.
Hans Deragon wrote:
Greetings M. Chu,
The most practical solution is to send running status messages to one
stream, as you suggest, though I would send it to stdout, not stderr. In
my view, stderr is for errors and diagnostics in the sense of debugging,
not for normal operation. A tool that reports the status of its
operations should send its output to stdout. Connection errors, timeouts
and such abnormal errors should be reported to stderr. Duplicate values,
existing values, entry not found and such are normal operational messages
and should be sent to stdout.
I made my changes because they were the simplest, fasted solutions which
do not modify the current tools default behavior. I also believe that in
the context of logging, using one single stream to record all events in
chronological order is very practical. Since we disagree on this matter
and you are chef architect of OpenLDAP, you decision stands and I respect
that. We will leave the tools as is.
You still have several other alternatives to investigate. As Hallvard
suggested, simply calling setlinebuf(stdout) in the tools may be sufficient.
Best regards,
Hans Deragon
> Hallvard Breien Furuseth wrote:
>> Hans Deragon wrote:
>>> I implemented a simple common option (-K) for the client tools that
>>> if
>>> enabled, sets the file descriptor used to output errors to stdout
>>> instead
>>> of the default (and currently used) stderr.
>
> That would be stupid. Error messages belong on stderr, that's why it
> exists.
>
>>> This is done because when running a batch of commands with the output
>>> sent
>>> to a file, logs collected are not shown chronologically because the
>>> output
>>> is split between stdout and stderr. (...)
>
> There are two obvious fixes:
> 1) don't send batches of commands that ignore errors, so that the
> processing immediately stops at the failed entry.
> 2) change the error message text to include the input line number where
> it
> failed.
>
> Also a possibility:
> don't send running status messages to stdout, send them only to stderr.
> Technically the only commandline tool that normally produces "output" is
> ldapsearch. Everything else is "diagnostic" and belongs on stderr anyway.
>
>> I suggest instead a -B option to make stdout unbuffered or
>> linebuffered.
>
> stdout is already linebuffered by default when directed to a tty.
>
> --
> -- Howard Chu
> CTO, Symas Corp. http://www.symas.com
> Director, Highland Sun http://highlandsun.com/hyc/
> Chief Architect, OpenLDAP http://www.openldap.org/project/
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
4045
days inactive
4062
days old
4 comments
3 participants
participants (3)
-
Hallvard Breien Furuseth -
Hans Deragon -
Howard Chu