Please read slapd(8) and slapd.conf(5) man pages, specifically slapd options -s -n -l.

I'm not sure why we wouldn't want syslog. Using the -d flag will change it to stderr, or you could modify the Debug() macro, if you don't think syslog is the right fit. I guess there's an argument for variable severity...of course that could still be done with syslog() and a bit of code.

loglevel sync will show you syncrepl stuff. Not quite as high level as you'd want, but it should be derivable. I think there is/was/might be some Perl script in contrib for log parsing. You might start there.

On Fri, 7 Dec 2007, Craig wrote:

It looks like a loglevel of 256 will give me the basic "client request data". (It's just not in one line. But, that's a 'parsing problem'. :> )

I still have the questions around using something other than local4. And, the possibility of using "local.error" vs "local.info".

Also, not sure how data from a sync is logged (at loglevel 256).

Lastly, there's the questions about future work: sticking with syslog? Putting different things in different files? (Error vs debug vs normal requests)

Thanx Craig

Craig wrote:

...

Hi, I was recently looking at our logs and trying to figure out what an appropriate logging level is for a stable, production system.

What I would really like is a log (or logs) that contain:

• the request made
• the client (IP) that made the request
• how much time it took to answer the request
• any errors, with LDAP error codes including errors with configs
• syncrepl info, eg: "sync completed added 2 entries, changed 4"

The current log level scheme doesn't seem to support that. (Please correct me if I'm wrong.) I guess I am looking for something more like Apache's logging (access/request log and an error log).

While openldap uses syslog, there is no mention of it supporting "debug/info/error/warn" type of log differentiation.

Is it possible to do all of the above "today"? If not, is it "on the plan"?

Is there any plan to move away from using syslog? Or at least, make it configurable which syslog facility to use? (Not having "local4" hardcoded.)

Any comments would be appreciated.

TIA! Craig

Howard Chu wrote:

There is no "plan" - OpenLDAP grows when an interested developer shows up and writes code that they're interested in. There was an effort to

That's fair. I only asked because it feels like most free SW does some kind of "custom logging". Syslog doesn't appear to be used as heavily as it once was. (That observation is purely anecdotal... I have no hard evidence.) I was just curious.

I think loglevel "stats" already provides most of what you want. Add

I was looking at some old docs that don't mention the "names", just the numbers. (That is exactly what I was looking for.)

problems crop up, with requests taking excessive time, that'll already be obvious in the regular syslog timestamps.

True. I wasn't looking for fine-grained timing data. Something more like "it takes 1 sec" vs "it takes 30 secs". So, the "rough" timing by syslog is just fine.

local4 is not hardcoded in current releases. If it's hardcoded in the version you're running, then whatever you're running is too old.

That comment was an inference from several web pages that I read. After Quanah's reply, I reread the man page for slapd and it's right there at the top.

Also, his comment about auditlog and accesslog overlays were cool. I'll look at those.

Thanx everyone for the help. (My apologies that most of it was RTFM.)

Craig

Howard Chu wrote:

When you're looking for a software feature, the manpages and Admin Guide should be your first resort. Pretty much every feature is documented.

This morning there were some posts about "rewriting overlays". So, I wanted to learn more about what they could do. I resisted the urge to use Google and went directly to the OpenLDAP Admin doc:

http://www.openldap.org/doc/admin24/

First, let me say that the docs do look really complete. But, this is good and bad. The bad part is that it is a little overwhelming. Can a search be added to the admin docs? I wanted to know more about a specific overlay and didn't see it in the table of contents. So, I didn't know where to start... Google is the next viable option, IMO.

The FAQ-o-matic is very nice, but I think an updated interface would help a great deal. (Not sure if the "faq-o-matic" package allows for easy changes to the interface. And I am NOT suggesting removing a perfectly good piece of software for something that looks nicer, but is less functional.)

Lastly, the man pages... Again, the size is a bit daunting. There are 78 man pages with 2.3.35. (With an additional 121 symlinked files.) That's quite a bit when you're looking for one specific thing and don't really know where to start. I want to be very clear; I am NOT knocking the docs at all. As I started looking around more, it is a lot more clear how things are laid out. But, when LDAP is just a tool and not a core part of my job, it is difficult to spend 2 hours reading docs for a feature we may not even need. I was just looking for a quick description. My hope is that my experience gives you more insight to what, at least, one sysadmin finds difficult. (If I am the minority, then prioritize my thoughts appropriately.)

When a feature in the documentation isn't clear enough to you, it's fine to ask on the mailing list, but even better is to submit an ITS pointing out exactly what isn't clear. Sometimes we see problems on the list that

So (just to be clear), you'd want me to file a bug for adding a search box to the admin docs? I looked at the bug pages and didn't see anything about searching the docs:

http://www.openldap.org/its/index.cgi/Documentation

On a side note, I noticed that jitterbug is no longer being maintained. Have you considered migrating to, say, Bugzilla? (I do realize how big of an undertaking that is, I am *just asking*. :> )

Thanx Craig

I was very hesitant to respond because flame wars are simply a waste of time and serve no purpose. However, I find it simply too condescending to let it go. Also, you posted your reply publicly, so I will as well.

Howard Chu wrote:

Craig wrote:

...

Lastly, the man pages... Again, the size is a bit daunting. There are 78 man pages with 2.3.35. (With an additional 121 symlinked files.) That's quite a bit when you're looking for one specific thing and don't really know where to start. I want to be very clear; I am NOT knocking the docs at all. As I started looking around more, it is a lot more clear how things are laid out. But, when LDAP is just a tool and not a core part of my job, it is difficult to spend 2 hours reading docs for a feature we may not even need. I was just looking for a quick description. My hope is that my experience gives you more insight to what, at least, one sysadmin finds difficult. (If I am the minority, then prioritize my thoughts appropriately.)

Aside from Gavin's reply re: apropos, there's also grep and other tools. I don't see that any of this is an obstacle to a Unix sysadmin. If it's

I explained in my reply to Gavin why that didn't work. Don't assume that every environment is set up the same way yours is... (We all have something in our environment we would like to fix, but don't have the time.) Also, don't assume that fixing man pages for one application that is working fine is anywhere near the top of my priority list. (I will accept criticism for not having the man pages in a spot where they are indexed or for forgetting that they are not in said spot. But, I will not accept statements that say I do or do not know how to do something when you clearly have no idea.)

A simple response of "man -k rwm" would have been perfect. (Which is exactly what Gavin did.) I would have then realized/remembered that my (or the "system") man path needs to be updated and that makewhatis needs to be run.

a challenge to you, I suggest you immediately go buy a copy of _The_Unix_Programming_Environment_ and read it.

If you are going to suggest a book, I would recommend choose something more relevant, or least something that has been updated in the past 20 years. That book is seriously out of date. Also, I believe there are several other books that are significantly better:

http://www.amazon.com/UNIX-System-Administration-Handbook-3rd/dp/0130206016/... http://www.oreilly.com/catalog/esa3/ http://www.oreilly.com/catalog/upt3/

When people complain that thus-and-such feature hasn't been documented, that's a legitimate flaw. I have no sympathy for people who complain that we have too much documentation. (Heck, I think that's even the first time anyone has ever made that complaint. Usually they're

I would strongly suggest you actually READ my email next time.

"I want to be very clear; I am NOT knocking the docs at all."

I really do NOT (do I have to say it again?!?!?! ***NOT***) see why this was so difficult to understand.

Again, learn how to find information efficiently. It's not that hard, but it's a prereq to being able to do your job at all.

Do not assume that you understand the first part of my job or what it entails. You know almost nothing about me, yet you know what the prerequisites are for my job, or even what exactly my job really is?!?! How many scientists do sysadmin work because they can't get a grant to cover hiring one?

...

On a side note, I noticed that jitterbug is no longer being maintained.

"Maintained" is an interesting word, in open source. The source code is available, and it's completely under our control. We've tailored it to do what we want it to do. As such, I don't think you can call it unmaintained.

I find this probably the most ironic of your entire message. You are critisizing me for, in essence, being lazy and not reading the docs or knowing how to find information. If you go to the OpenLDAP ITS main page;

http://www.openldap.org/its/

There is a link for Jitterbug at the bottom. (I only point this out, because I don't know if this link is the "correct one". It is simply the one list on your web pages.)

http://www.samba.org/jitterbug/

At the top of that page, there is the following statement:

JitterBug project suspended

The JitterBug project is no longer being actively maintained.

Realizing that people are busy, I was *merely* pointing this fact out. Nothing more.

I want to may sure this is clear; my original message was simply saying what *I* found difficult. I even said: "If I am the minority, then prioritize my thoughts appropriately."

In the end, I was asking a pretty simple question and letting you know what I found difficult. I was very clear that I was NOT (again, let me say it twice so maybe you will actually *SEE* it... ***NOT***) criticizing the docs or the web pages. I was just trying to suggest a possible improvement.

In the future, I will be much more careful about giving any kind of feedback to the OpenLDAP community.

Craig

There is no "plan" - OpenLDAP grows when an interested developer shows up and writes code that they're interested in. There was an effort to introduce various log levels back in the OpenLDAP 2.1 timeframe but it turned out to be too cumbersome and we eventually gave up on it. If someone else wants to take a stab at it, they're welcome to give it a go.

During the lifecycle of OpenLDAP 2.3, we were requested by a customer to provide syslog level capability in custom code. The request sounded fair enough to result in a general feature that eventually made it into 2.4 (the "Log" macro family). However, at that time,

- resources (read: money and time) were too short for the thousands of messages in OpenLDAP sources

- a change of the whole code would have required consensus in general, and consensus on the most appropriate level for each message, or at least for each message type, and we couldn't guarantee this within the time constraints of that work.

So this feature was wrapped by the old "Debug" macro to preserve the original behavior. If anybody feels like classifying the messages, seeking consensus on syslog levels, and modifying the code accordingly, I think this effort would be more than welcome. As a side effect, it would allow to get rid of the format mismatches when compiling with -Wall :)

I think loglevel "stats" already provides most of what you want. Add loglevel "sync" to that and you should be good to go.

Probably "sync" is too analytic for most needs, it could be split into an "sync-stats" level and a "sync-trace" level (the former could just be triggered by "stats"?), but this is getting OT for this answer

p.

Ing. Pierangelo Masarati OpenLDAP Core Team

SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati@sys-net.it ---------------------------------------

Pierangelo Masarati wrote:

...

There is no "plan" - OpenLDAP grows when an interested developer shows up and writes code that they're interested in. There was an effort to introduce various log levels back in the OpenLDAP 2.1 timeframe but it turned out to be too cumbersome and we eventually gave up on it. If someone else wants to take a stab at it, they're welcome to give it a go.

During the lifecycle of OpenLDAP 2.3, we were requested by a customer to provide syslog level capability in custom code. The request sounded fair enough to result in a general feature that eventually made it into 2.4 (the "Log" macro family).

Ah right, forgot about that.

However, at that time,

• resources (read: money and time) were too short for the thousands of

messages in OpenLDAP sources

• a change of the whole code would have required consensus in general, and

consensus on the most appropriate level for each message, or at least for each message type, and we couldn't guarantee this within the time constraints of that work.

So this feature was wrapped by the old "Debug" macro to preserve the original behavior. If anybody feels like classifying the messages, seeking consensus on syslog levels, and modifying the code accordingly, I think this effort would be more than welcome. As a side effect, it would allow to get rid of the format mismatches when compiling with -Wall :)

Yes, would be nice, but it's more than just reclassifying existing usage. LDAP_DEBUG_TRACE is far overused since it is effectively a level and not a proper subsystem. Eliminating it would require the introduction of many new subsystems. (Of course some of the experience from the earlier attempt may still apply here.) Likewise LDAP_DEBUG_ANY is used for all errors; the point is that these messages are important no matter what subsystems you originally requested. LDAP_DEBUG_NONE should probably have been named LDAP_DEBUG_ERROR instead since that's what it's really used for on the commandline.

Our commandline and config syntax would need to change to specify logging pairs instead of simple keywords or bitmasks; e.g. "-d config.warning" (much like syslog itself).

One thing I think we've learned by now, migrating this is bigger than a summer student project.

Buchan Milne wrote:

On Saturday 08 December 2007 01:42:02 Craig wrote:

...

• how much time it took to answer the request

IMHO you should avoid using a logging system as a performance monitoring system.

I didn't mean to say that I was looking at logging for fine-grained performance analysis. I only wanted something that would show rough numbers to signify that there was a possible problem. I.e. it takes 60 seconds to answer some particular request. That is either a sign of the server being overloaded or that the client is sending an inefficient query.

can be done with a monitoring tool that monitors the contextCSNs. I have a script that does that for use with hobbit:

http://staff.telkomsa.net/~bgmilne/hobbit/

Thanx for the pointer.

...

The current log level scheme doesn't seem to support that.

seems to. Of course, whether this is the most sensible way to achieve what you

Unfortunately, I was looking at some old documentation (web page).

want, we can't know, as you only told us how you want to do it (parse logs), not what you are trying to achieve (understand performance, ensure replication is occurring and that slaves are in sync?).

I feel the rest of your comments on logging are irrelevant until you tell us what you are trying to achieve (as logging might not be the best method).

I was simply looking for some advice on how to configure logging to get information X,Y,Z. My only comment about parsing was that I didn't care about the format, I just cared about the data.

Our LDAP servers are working just fine. I was looking at the logs and they were a little more verbose in some areas and less in others. I just wanted the logs have enough information so that if I had a problem I could start to debug what was going on. Yet, I don't want "logging" to start affecting performance.