exec newslink.js 1.108 1.109
Update of /cvsroot/sbbs/exec
In directory cvs:/tmp/cvs-serv23562

Modified Files:
newslink.js
Log Message:
Created a new function (save_ptr) to elminate some redundant code.
Removed support for the long-deprecated .snl import/export pointer files
(uses the long-supported .ini file now only, [NewsLink] section).
Added more debug output (enabled with the -d switch) when skipping
(not exporting) any locally posted messages to help debug Bill McGarrity's reported problem.


---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

Hiya Rob...

rswindell wrote to CVS commit on 05-09-18 11:48 <=-

exec newslink.js 1.108 1.109
Update of /cvsroot/sbbs/exec
In directory cvs:/tmp/cvs-serv23562

Modified Files:
newslink.js
Log Message:
Created a new function (save_ptr) to elminate some redundant code.
Removed support for the long-deprecated .snl import/export pointer
files (uses the long-supported .ini file now only, [NewsLink] section). Added more debug output (enabled with the -d switch) when skipping
(not exporting) any locally posted messages to help debug Bill
McGarrity's reported problem.

OK... downloaded the new newslink.js file and ran a test. I created a message in MultiMail and uploaded the QWK pkt to the system and manually ran Newslink. I must also state the USENET areas, I'm gating between FTN and usenet. When I uploaded the file, FIDOOUT ran instantly. As stated, I then ran newlink manually with the -d switch. Here are the results:

5/9 07:36:20p sub: usenet_altbbsfg, newsgroup: alt.sports.baseball.sf-giants
5/9 07:36:20p usenet_altbbsfg.ini export ptr: 13064
5/9 07:36:20p usenet_altbbsfg.ini import ptr: 331880
5/9 07:36:20p exporting local messages
5/9 07:36:20p skipping msg from the Internet #13065
5/9 07:36:20p skipping msg from the Internet #13066
5/9 07:36:20p cmd: POST
5/9 07:36:20p rsp: 340 send article
5/9 07:36:20p cmd: From: "Bill McGarrity" <bill.mcgarrity@tequilamockingbirdonline.net.remove-9r2-this>
5/9 07:36:20p cmd: To: All
5/9 07:36:20p cmd: Subject: Test message...
5/9 07:36:20p cmd: Message-ID: undefined
5/9 07:36:20p cmd: Date: Wed, 9 May 2018 19:33:00 -0400
5/9 07:36:20p cmd: X-Comment-To: All
5/9 07:36:20p cmd: Organization: TequilaMockingbird Online
5/9 07:36:20p cmd: Newsgroups: alt.sports.baseball.sf-giants
5/9 07:36:20p cmd: Content-Type: text/plain; charset=IBM437
5/9 07:36:20p cmd: Content-Transfer-Encoding: 8bit
5/9 07:36:20p cmd: X-Gateway: tequilamockingbirdonline.net [Synchronet 3.17a-Win32 NewsLink 1.109]
5/9 07:36:20p cmd:
5/9 07:36:20p cmd: .
5/9 07:36:20p rsp: 441 posting failed
5/9 07:36:20p !POST FAILURE: 441 posting failed
5/9 07:36:20p cmd: GROUP alt.sports.baseball.sf-giants
5/9 07:36:21p rsp: 211 76 331805 331880 alt.sports.baseball.sf-giants

Here is the msg header:

To All
Subject Test message...
SenderExt 1
Sender Bill McGarrity
when_written 5AF3852C C12C Wed May 09 2018 19:33:00 EDT
when_imported 5AF385D3 C12C Wed May 09 2018 19:35:47 EDT
type 0000h
version 0121h
attr 0000h
auxattr 00000000h
netattr 00000000h
header offset 289720h
header length 139
number 13067
thread_id 13067
data offset 24B200h
data field[0] TEXT_BODY, offset 0, length 410
data field[1] TEXT_TAIL, offset 410, length 27


I then posted locally on the bbs and saved the message. Again, FIDOOUT ran right after saving the message. I then manually ran newslink and here are the results:

5/9 07:50:00p exporting local messages
5/9 07:50:00p cmd: POST
5/9 07:50:00p rsp: 340 send article
5/9 07:50:00p cmd: From: "Bill McGarrity" <bill.mcgarrity@tequilamockingbirdonline.net.remove-vwt-this>
5/9 07:50:00p cmd: To: All
5/9 07:50:00p cmd: Subject: Test message..
5/9 07:50:00p cmd: Message-ID: <5AF38902.13068.usenet_altbbsfg@tequilamockingbirdonline.net>
5/9 07:50:00p cmd: Date: Wed, 9 May 2018 19:49:22 -0400
5/9 07:50:00p cmd: X-Comment-To: All
5/9 07:50:00p cmd: Organization: TequilaMockingbird Online
5/9 07:50:00p cmd: Newsgroups: alt.sports.baseball.sf-giants
5/9 07:50:00p cmd: X-FTN-PID: Synchronet 3.17a-Win32 Apr 30 2018 MSC 1800
5/9 07:50:00p cmd: X-FTN-MSGID: 13068.usenet_altbbsfg@1:266/404 1f58f20e
5/9 07:50:00p cmd: Content-Type: text/plain; charset=IBM437
5/9 07:50:00p cmd: Content-Transfer-Encoding: 8bit
5/9 07:50:00p cmd: X-Gateway: tequilamockingbirdonline.net [Synchronet 3.17a-Win32 NewsLink 1.109]
5/9 07:50:00p cmd:
5/9 07:50:00p cmd: .
5/9 07:50:00p rsp: 240 post accepted.


Here is the successful msg header:

To All
Sender Bill McGarrity
SenderExt 1
SenderUserID Bill McGarrity
SenderTime 20180509233510Z
SenderIpAddr 127.0.0.1
SenderHostName HomeServer.tequilamockingbirdonline.net
SenderProtocol Telnet
SenderPort 60605
SenderServer tequilamockingbirdonline.net
Subject Test message..
Message-ID <5AF38902.13068.usenet_altbbsfg@tequilamockingbirdonline.net> X-FTN-MSGID 13068.usenet_altbbsfg@1:266/404 1f58f20e
X-FTN-PID Synchronet 3.17a-Win32 Apr 30 2018 MSC 1800
Editor Open!EDIT
when_written 5AF38902 C12C Wed May 09 2018 19:49:22 EDT
when_imported 5AF38902 C12C Wed May 09 2018 19:49:22 EDT
type 0000h
version 0121h
attr 0000h
auxattr 00000000h
netattr 00000000h
header offset 289820h
header length 442
number 13068
thread_id 13068
data offset 24B400h
data field[0] TEXT_BODY, offset 0, length 387


I hope this helps....




--

Bill

Telnet: tequilamockingbirdonline.net
Web: bbs.tequilamockingbirdonline.net
FTP: ftp.tequilamockingbirdonline.net:2121
IRC: irc.tequilamockingbirdonline.net Ports: 6661-6670 SSL: +6697
Radio: radio.tequilamockingbirdonline.net:8010/live


... Look Twice... Save a Life!!! Motorcycles are Everywhere!!!
--- MultiMail/Win32 v0.50
þ Synchronet þ TequilaMockingbird Online - Toms River, NJ

Re: exec/newslink.js
By: Bill McGarrity to rswindell on Wed May 09 2018 07:56 pm

5/9 07:36:20p cmd: Message-ID: undefined

I think that ^^^

5/9 07:36:20p !POST FAILURE: 441 posting failed

Is causing this ^^^

So... there's no message-ID stored with the message and one isn't being dynamically generated when newslink reads the msg-header. I'll look into it.


digital man

This Is Spinal Tap quote #41:
Ian Faith: It say's "Memphis show cancelled due to lack of advertising funds." Norco, CA WX: 73.9øF, 54.0% humidity, 11 mph NE wind, 0.00 inches rain/24hrs

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

Re: exec/newslink.js
By: Digital Man to Bill McGarrity on Wed May 09 2018 06:55 pm

Re: exec/newslink.js
By: Bill McGarrity to rswindell on Wed May 09 2018 07:56 pm

5/9 07:36:20p cmd: Message-ID: undefined

I think that ^^^

5/9 07:36:20p !POST FAILURE: 441 posting failed

Is causing this ^^^

So... there's no message-ID stored with the message and one isn't being dynamically generated when newslink reads the msg-header. I'll look into it.

I can't seem to reproduce what you're seeing. It's somewhat normal to have a message with no message-ID and when the header of such a message is read via JavaScript (e.g. newslink.js), the message-ID is dynamically generated. I tested this using a little test script and it worked fine. What version of Synchronet are you using? Are you running newslink via jsexec or as an event in Synchronet?

digital man

Synchronet "Real Fact" #84:
The Electronic Frontier Foundation used to run Synchronet (circa 1993).
Norco, CA WX: 67.5øF, 64.0% humidity, 4 mph E wind, 0.00 inches rain/24hrs

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

Digital Man wrote to Bill McGarrity on 05-09-18 18:55 <=-

Re: exec/newslink.js
By: Bill McGarrity to rswindell on Wed May 09 2018 07:56 pm

5/9 07:36:20p cmd: Message-ID: undefined

I think that ^^^

5/9 07:36:20p !POST FAILURE: 441 posting failed

Is causing this ^^^

So... there's no message-ID stored with the message and one isn't being dynamically generated when newslink reads the msg-header. I'll look
into it.

Thank you...


--

Bill

Telnet: tequilamockingbirdonline.net
Web: bbs.tequilamockingbirdonline.net
FTP: ftp.tequilamockingbirdonline.net:2121
IRC: irc.tequilamockingbirdonline.net Ports: 6661-6670 SSL: +6697
Radio: radio.tequilamockingbirdonline.net:8010/live


... Look Twice... Save a Life!!! Motorcycles are Everywhere!!!
--- MultiMail/Win32 v0.50
þ Synchronet þ TequilaMockingbird Online - Toms River, NJ

Digital Man wrote to Bill McGarrity on 05-09-18 20:37 <=-

Re: exec/newslink.js
By: Digital Man to Bill McGarrity on Wed May 09 2018 06:55 pm

Re: exec/newslink.js
By: Bill McGarrity to rswindell on Wed May 09 2018 07:56 pm

5/9 07:36:20p cmd: Message-ID: undefined

I think that ^^^

5/9 07:36:20p !POST FAILURE: 441 posting failed

Is causing this ^^^

So... there's no message-ID stored with the message and one isn't being dynamically generated when newslink reads the msg-header. I'll look into it.

I can't seem to reproduce what you're seeing. It's somewhat normal to
have a message with no message-ID and when the header of such a message
is read via JavaScript (e.g. newslink.js), the message-ID is
dynamically generated. I tested this using a little test script and it worked fine. What version of Synchronet are you using? Are you running newslink via jsexec or as an event in Synchronet?

Here are the versions of Synchronet:

Synchronet Terminal Server 3.17a SMBLIB 2.60 Compiled Apr 30 2018 05:02:26 with MSC 1800
Synchronet Mail Server 1.672 SMBLIB 2.60 Compiled Apr 30 2018 05:03:24 with MSC 1800
Synchronet FTP Server 1.472 Compiled Apr 30 2018 05:03:16 with MSC 1800 Synchronet Web Server 1.667 Compiled Apr 30 2018 05:03:39 with MSC 1800 Synchronet Services 1.322 Compiled Apr 30 2018 05:03:36 with MSC 1800 Synchronet Control Panel v3.17.0.0 Compiled Apr 30 2018 05:04:44 with BCC 5.60 Synchronet Local Spy ANSI Terminal Emulation TEmulVT (c) 1996-2002 F. Piette V2.19
JavaScript-C 1.8.5 2011-03-31 (c) 1998 Netscape Communications Corp.
Cryptlib v3.4.4.0 Copyright 1992-2006 Peter Gutmann. All rights reserved.

The NEWSLINK is being run as a timed event.

Internal Code NEWSLINK
Start-up Directory
Command Line ?newslink.js -d
Enabled Yes
Execution Node 10
Execution Months Any
Execution Days of Month Any
Execution Days of Week All
Execution Frequency 96 times a day
Requires Exclusive Execution No
Force Users Off-line For Event No
Native Executable No
Use Shell to Execute No
Background Execution No
Always Run After Init/Re-init No

JS files:

// $Id: newslink.js,v 1.109 2018/05/09 18:48:58 rswindell Exp $
// $Id: newsutil.js,v 1.28 2017/10/31 19:44:46 rswindell Exp $

Thanks...




--

Bill

Telnet: tequilamockingbirdonline.net
Web: bbs.tequilamockingbirdonline.net
FTP: ftp.tequilamockingbirdonline.net:2121
IRC: irc.tequilamockingbirdonline.net Ports: 6661-6670 SSL: +6697
Radio: radio.tequilamockingbirdonline.net:8010/live


... Look Twice... Save a Life!!! Motorcycles are Everywhere!!!
--- MultiMail/Win32 v0.50
þ Synchronet þ TequilaMockingbird Online - Toms River, NJ

Re: exec/newslink.js
By: Bill McGarrity to Digital Man on Thu May 10 2018 12:12 am

I can't seem to reproduce what you're seeing. It's somewhat normal to have a message with no message-ID and when the header of such a message is read via JavaScript (e.g. newslink.js), the message-ID is dynamically generated. I tested this using a little test script and it worked fine. What version of Synchronet are you using? Are you running newslink via jsexec or as an event in Synchronet?

Here are the versions of Synchronet:

Synchronet Terminal Server 3.17a SMBLIB 2.60 Compiled Apr 30 2018 05:02:26

I just committed to CVS a test script, msgutil.js. Can you run that on the message base in question and post the header output for the problem message (the one with the "undefined" message-ID)?

When I use this script to show the headers of a message base which contains a message with no message-ID, the "id" property value is dynamically generated (a valid RFC-822 compliant message ID is created). NewsLink (newslink.js) uses the exact same method for retrieving the message header and should be getting the same result.

You can run msgutil.js using jsexec like so: "jsexec msgutil <internal-code>" where <internal-code> is the internal code of the msgbase in question. Right now, this script just dumps all the index and header files for all the messages in the base.

digital man

Synchronet "Real Fact" #33:
The Synchronet web user interface was contributed by Robert Couture, Runemaster.
Norco, CA WX: 76.4øF, 54.0% humidity, 4 mph NE wind, 0.00 inches rain/24hrs

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

Re: exec/newslink.js
By: Digital Man to Bill McGarrity on Thu May 10 2018 12:10:49

HIya Rob...

I just committed to CVS a test script, msgutil.js. Can you run that
on the message base in question and post the header output for the
problem message (the one with the "undefined" message-ID)?

When I use this script to show the headers of a message base which contains a message with no message-ID, the "id" property value is dynamically generated (a valid RFC-822 compliant message ID is
created). NewsLink (newslink.js) uses the exact same method for
retrieving the message header and should be getting the same result.

You can run msgutil.js using jsexec like so: "jsexec msgutil <internal-code>" where <internal-code> is the internal code of the
msgbase in question. Right now, this script just dumps all the index
and header files for all the messages in the base.

OK... done. Here is the output on the message uploaded in the REP pkt:

13067
total_votes = 0
number = 13067
offset = 0
to = All
from = Bill McGarrity
subject = Test message...
from_ext = 1
to_agent = 0
from_agent = 0
replyto_agent = 0
to_net_type = 0
from_net_type = 0
replyto_net_type = 0
forwarded = 0
expiration = 0
priority = 0
cost = 0
type = 0
version = 289
attr = 0
auxattr = 0
netattr = 0
when_written_time = 1525908780
when_written_zone = -16084
when_written_zone_offset = -240
when_imported_time = 1525908947
when_imported_zone = -16084
when_imported_zone_offset = -240
thread_id = 13067
thread_back = 0
thread_next = 0
thread_first = 0
delivery_attempts = 0
last_downloaded = 0
times_downloaded = 0
data_length = 437
date = Wed, 9 May 2018 19:33:00 -0400
votes = 0
id = <5AF385D3.13067.usenet_altbbsfg@tequilamockingbirdon
line.net>

Here is the msg that I wrote while online:

13068
total_votes = 0
number = 13068
offset = 0
to = All
from = Bill McGarrity
subject = Test message..
from_ext = 1
to_agent = 0
from_agent = 0
replyto_agent = 0
to_net_type = 0
from_net_type = 0
replyto_net_type = 0
from_ip_addr = 127.0.0.1
from_host_name = HomeServer.tequilamockingbirdonline.net
from_protocol = Telnet
from_port = 60605
sender_userid = Bill McGarrity
sender_server = tequilamockingbirdonline.net
sender_time = 20180509233510Z
forwarded = 0
expiration = 0
priority = 0
cost = 0
type = 0
version = 289
attr = 512
auxattr = 0
netattr = 0
when_written_time = 1525909762
when_written_zone = -16084
when_written_zone_offset = -240
when_imported_time = 1525909762
when_imported_zone = -16084
when_imported_zone_offset = -240
thread_id = 13068
thread_back = 0
thread_next = 0
thread_first = 13078
delivery_attempts = 0
last_downloaded = 0
times_downloaded = 0
data_length = 387
date = Wed, 9 May 2018 19:49:22 -0400
votes = 0
id = <5AF38902.13068.usenet_altbbsfg@tequilamockingbirdon
line.net>
ftn_msgid = 13068.usenet_altbbsfg@1:266/404 1f58f20e
ftn_pid = Synchronet 3.17a-Win32 Apr 30 2018 MSC 1800
field_list =
can_read = true

Now as I orginally stated, I gate this to both FTN and UUCP. FIDOOUT ran on both occasions (when u/l in REP pkt and when saved online.) As you can
see, the first doesn't have any FTM properties as well.

Hope this helps....

--

Bill

Telnet: tequilamockingbirdonline.net
Web: bbs.tequilamockingbirdonline.net
FTP: ftp.tequilamockingbirdonline.net:2121
IRC: irc.tequilamockingbirdonline.net Ports: 6661-6670 SSL: 6697
Radio: radio.tequilamockingbirdonline.net:8010/live

---
þ Synchronet þ TequilaMockingbird Online - Toms River, NJ

Re: exec/newslink.js
By: Bill McGarrity to Digital Man on Thu May 10 2018 04:49 pm

OK... done. Here is the output on the message uploaded in the REP pkt:

13067
id = <5AF385D3.13067.usenet_altbbsfg@tequilamockingbirdon
line.net>

Here is the msg that I wrote while online:

13068
id = <5AF38902.13068.usenet_altbbsfg@tequilamockingbirdon
line.net>

Okay, cool, in both cases, the JS msg header includes the "id" field which is what is used to populate the NNTP article "Message-ID" header field.

Now as I orginally stated, I gate this to both FTN and UUCP. FIDOOUT ran on both occasions (when u/l in REP pkt and when saved online.) As you can
see, the first doesn't have any FTM properties as well.

That shouldn't matter: The FTN header fields/properties aren't required for NNTP article headers.

So the next question... if you run newslink.js using jsexec, do you have the same problem (the "undefined" Message-ID)?

digital man

Synchronet "Real Fact" #79:
172 Synchronet Match Maker registrations were sold (@$69) between 1995 and 1996.
Norco, CA WX: 77.0øF, 50.0% humidity, 14 mph NE wind, 0.00 inches rain/24hrs

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

Digital Man wrote to Bill McGarrity on 05-10-18 14:34 <=-

Re: exec/newslink.js
By: Bill McGarrity to Digital Man on Thu May 10 2018 04:49 pm

OK... done. Here is the output on the message uploaded in the REP pkt:

13067
id = <5AF385D3.13067.usenet_altbbsfg@tequilamockingbirdon
line.net>

Here is the msg that I wrote while online:

13068
id = <5AF38902.13068.usenet_altbbsfg@tequilamockingbirdon
line.net>

Okay, cool, in both cases, the JS msg header includes the "id" field
which is what is used to populate the NNTP article "Message-ID" header field.

Good...

Now as I orginally stated, I gate this to both FTN and UUCP. FIDOOUT ran on both occasions (when u/l in REP pkt and when saved online.) As you can
see, the first doesn't have any FTM properties as well.

That shouldn't matter: The FTN header fields/properties aren't required for NNTP article headers.

OK...

So the next question... if you run newslink.js using jsexec, do you
have the same problem (the "undefined" Message-ID)?

OK... I uploaded a REP pkt and ran jsexec newslink.js -d

Here is the output for that sub...

!IMPORT 81351 ERROR: smb_addmsg duplicate Message-ID:
pb1aiq$29s$1@gioia.aioe.o

found in message #64

sub: usenet_altbbsfg, newsgroup: alt.sports.baseball.sf-giants usenet_altbbsfg.ini export ptr: 13083
usenet_altbbsfg.ini import ptr: 331895
exporting local messages
cmd: POST
rsp: 340 send article
cmd: From: "Bill McGarrity"
bill.mcgarrity@tequilamockingbirdonline.net.remove-
p1y-this>
cmd: To: All
cmd: Subject: One more test...
cmd: Message-ID: undefined
cmd: Date: Thu, 10 May 2018 20:59:00 -0400
cmd: X-Comment-To: All
cmd: Organization: TequilaMockingbird Online
cmd: Newsgroups: alt.sports.baseball.sf-giants
cmd: Content-Type: text/plain; charset=IBM437
cmd: Content-Transfer-Encoding: 8bit
cmd: X-Gateway: tequilamockingbirdonline.net [Synchronet 3.17a-Win32 NewsLink
.
109]
cmd:
cmd: .
rsp: 441 posting failed
!POST FAILURE: 441 posting failed
cmd: GROUP alt.sports.baseball.sf-giants
rsp: 211 76 331820 331895 alt.sports.baseball.sf-giants alt.sports.baseball.sf-giants import ptr: 331895, first_msg: 331820, last_msg:

31895

Synchronet NewsLink 1.109 session complete (0 exported, 0 imported) c:\sbbs\exec\newslink.js executed in 60.35 seconds

As you can see it wasn't exported.

Here is the output on the above msg from msgutil.js:

13084
total_votes = 0
number = 13084
offset = 0
to = All
from = Bill McGarrity
subject = One more test...
from_ext = 1
to_agent = 0
from_agent = 0
replyto_agent = 0
to_net_type = 0
from_net_type = 0
replyto_net_type = 0
forwarded = 0
expiration = 0
priority = 0
cost = 0
type = 0
version = 289
attr = 0
auxattr = 0
netattr = 0
when_written_time = 1526000340
when_written_zone = -16084
when_written_zone_offset = -240
when_imported_time = 1526000405
when_imported_zone = -16084
when_imported_zone_offset = -240
thread_id = 13084
thread_back = 0
thread_next = 0
thread_first = 0
delivery_attempts = 0
last_downloaded = 0
times_downloaded = 0
data_length = 375
date = Thu, 10 May 2018 20:59:00 -0400
votes = 0
id =
5AF4EB15.13084.usenet_altbbsfg@tequilamockingbirdon
line.net>
field_list =
can_read = true



Hope this helps...


--

Bill

Telnet: tequilamockingbirdonline.net
Web: bbs.tequilamockingbirdonline.net
FTP: ftp.tequilamockingbirdonline.net:2121
IRC: irc.tequilamockingbirdonline.net Ports: 6661-6670 SSL: +6697
Radio: radio.tequilamockingbirdonline.net:8010/live


... Look Twice... Save a Life!!! Motorcycles are Everywhere!!!
--- MultiMail/Win32 v0.50
þ Synchronet þ TequilaMockingbird Online - Toms River, NJ

exec newslink.js 1.109 1.110
Update of /cvsroot/sbbs/exec
In directory cvs:/tmp/cvs-serv22638

Modified Files:
newslink.js
Log Message:
If msgbase.get_msg_header() returns a header with no (undefined) message-ID header field, print/log an error and continue on (don't attempt to send to
NNTP server). This won't solve Bill McGarrity's problem, but it might
provide a clue (?).



---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

Re: exec/newslink.js
By: Bill McGarrity to Digital Man on Thu May 10 2018 09:14 pm

OK... I uploaded a REP pkt and ran jsexec newslink.js -d

Here is the output for that sub...

sub: usenet_altbbsfg, newsgroup: alt.sports.baseball.sf-giants usenet_altbbsfg.ini export ptr: 13083
usenet_altbbsfg.ini import ptr: 331895
exporting local messages
cmd: POST
rsp: 340 send article
cmd: From: "Bill McGarrity" bill.mcgarrity@tequilamockingbirdonline.net.remove-
p1y-this>
cmd: To: All
cmd: Subject: One more test...
cmd: Message-ID: undefined
cmd: Date: Thu, 10 May 2018 20:59:00 -0400
cmd: X-Comment-To: All
cmd: Organization: TequilaMockingbird Online
cmd: Newsgroups: alt.sports.baseball.sf-giants
cmd: Content-Type: text/plain; charset=IBM437
cmd: Content-Transfer-Encoding: 8bit
cmd: X-Gateway: tequilamockingbirdonline.net [Synchronet 3.17a-Win32 NewsLink
.
109]
cmd:
cmd: .
rsp: 441 posting failed
!POST FAILURE: 441 posting failed
cmd: GROUP alt.sports.baseball.sf-giants
rsp: 211 76 331820 331895 alt.sports.baseball.sf-giants alt.sports.baseball.sf-giants import ptr: 331895, first_msg: 331820, last_msg:

31895

Synchronet NewsLink 1.109 session complete (0 exported, 0 imported) c:\sbbs\exec\newslink.js executed in 60.35 seconds

As you can see it wasn't exported.

Well, it actually was exported - it was just rejected by the NNTP server because the Message-ID was not formatted correctly ("undefined" is not a valid NNTP-compliant message-ID).

Here is the output on the above msg from msgutil.js:

13084

id = 5AF4EB15.13084.usenet_altbbsfg@tequilamockingbirdon
line.net>

There it is (and it's not "undefined" or absent/missing). So strange. I've updated newslink.js to print/log a warning message if it encounters local messages without Message-IDs and doesn't try to send them to the NNTP server. I don't think that's going to fix the problem, but at least it should confirm that the ID isn't coming from the JS object model as the string "undefined".

I currently have no theories why newslink.js would retrieve a missing/undefined 'id' property in the message header object yet msgutil.js gets the (dynamically-generated) header 'id' field/property just fine.

I expect when you run the updated newslink.js, you should see a printed/logged message like "Message #13085 is missing a Message-ID header field", based on the reported error. Now, *why* the property is missing is still a mystery to me. Please let me know.

digital man

Synchronet/BBS Terminology Definition #48:
SMTP = Simple Message Transfer Protocol
Norco, CA WX: 69.3øF, 55.0% humidity, 19 mph E wind, 0.00 inches rain/24hrs

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

Digital Man wrote to Bill McGarrity on 05-14-18 16:45 <=-

I currently have no theories why newslink.js would retrieve a missing/undefined 'id' property in the message header object yet msgutil.js gets the (dynamically-generated) header 'id' field/property just fine.

I expect when you run the updated newslink.js, you should see a printed/logged message like "Message #13085 is missing a Message-ID
header field", based on the reported error. Now, *why* the property is missing is still a mystery to me. Please let me know.

That's exactly the message I received....

5/15 01:54:10a sub: usenet_altbbsfg, newsgroup: alt.sports.baseball.sf-giants
5/15 01:54:10a usenet_altbbsfg.ini export ptr: 13125
5/15 01:54:10a usenet_altbbsfg.ini import ptr: 331934
5/15 01:54:10a Message #13126 is missing a Message-ID header field
5/15 01:54:10a alt.sports.baseball.sf-giants import ptr: 331934, first_msg: 331859, last_msg: 331934

Here is the info from msgutil.js:

13125
total_votes = 0
number = 13125
offset = 0
to = All
from = Bill McGarrity
subject = Another test... :/
from_ext = 1
to_agent = 0
from_agent = 0
replyto_agent = 0
to_net_type = 0
from_net_type = 0
replyto_net_type = 0
forwarded = 0
expiration = 0
priority = 0
cost = 0
type = 0
version = 289
attr = 0
auxattr = 0
netattr = 0
when_written_time = 1526362620
when_written_zone = -16084
when_written_zone_offset = -240
when_imported_time = 1526362642
when_imported_zone = -16084
when_imported_zone_offset = -240
thread_id = 13125
thread_back = 0
thread_next = 0
thread_first = 0
delivery_attempts = 0
last_downloaded = 0
times_downloaded = 0
data_length = 456
date = Tue, 15 May 2018 01:37:00 -0400
votes = 0
id =
5AFA7212.13125.usenet_altbbsfg@tequilamockingbirdon
line.net>
field_list =
can_read = true

Here is the message header from the sub-board operator menu:

To All
Subject Another test... :/
SenderExt 1
Sender Bill McGarrity
when_written 5AFA71FC C12C Tue May 15 2018 01:37:00 EDT
when_imported 5AFA7212 C12C Tue May 15 2018 01:37:22 EDT
type 0000h
version 0121h
attr 0000h
auxattr 00000000h
netattr 00000000h
header offset 297F20h
header length 142
number 13125
thread_id 13125
data offset 25A700h
data field[0] TEXT_BODY, offset 0, length 429
data field[1] TEXT_TAIL, offset 429, length 27


Here is a test message I created while logged into the system:

5/15 02:05:32a sub: usenet_altbbsfg, newsgroup:
lt.sports.baseball.sf-giants
5/15 02:05:32a usenet_altbbsfg.ini export ptr: 13126
5/15 02:05:32a usenet_altbbsfg.ini import ptr: 331934
5/15 02:05:32a Exported message 13127 to newsgroup: alt.sports.baseball.sf-giants
5/15 02:05:32a alt.sports.baseball.sf-giants import ptr: 331934, first_msg: 331860, last_msg: 331935
5/15 02:05:32a Getting headers for articles 331935 through 331935
5/15 02:05:32a 1 new articles
5/15 02:05:32a Retrieving article: 331935
5/15 02:05:33a Message 331935 imported into usenet_altbbsfg (1 of 1 total) 33 lines

msgutil.js info:

13127
total_votes = 0
number = 13127
offset = 0
to = All
from = Bill McGarrity
subject = One more test message..
from_ext = 1
to_agent = 0
from_agent = 0
replyto_agent = 0
to_net_type = 0
from_net_type = 0
replyto_net_type = 0
from_ip_addr = 127.0.0.1
from_host_name = HomeServer.tequilamockingbirdonline.net
from_protocol = Telnet
from_port = 64021
sender_userid = Bill McGarrity
sender_server = tequilamockingbirdonline.net
sender_time = 20180515055140Z
forwarded = 0
expiration = 0
priority = 0
cost = 0
type = 0
version = 289
attr = 0
auxattr = 0
netattr = 0
when_written_time = 1526363957
when_written_zone = -16084
when_written_zone_offset = -240
when_imported_time = 1526363957
when_imported_zone = -16084
when_imported_zone_offset = -240
thread_id = 13127
thread_back = 0
thread_next = 0
thread_first = 0
delivery_attempts = 0
last_downloaded = 0
times_downloaded = 0
data_length = 317
date = Tue, 15 May 2018 01:59:17 -0400
votes = 0
id =
5AFA7735.13127.usenet_altbbsfg@tequilamockingbirdon
line.net>
ftn_msgid = 13127.usenet_altbbsfg@1:266/404 1f5fe07c
ftn_pid = Synchronet 3.17a-Win32 Apr 30 2018 MSC 1800
field_list =
can_read = true

Header from the sub-board operator menu:

To All
Sender Bill McGarrity
SenderExt 1
SenderUserID Bill McGarrity
SenderTime 20180515055140Z
SenderIpAddr 127.0.0.1
SenderHostName HomeServer.tequilamockingbirdonline.net
SenderProtocol Telnet
SenderPort 64021
SenderServer tequilamockingbirdonline.net
Subject One more test message..
Message-ID <5AFA7735.13127.usenet_altbbsfg@tequilamockingbirdonline.net> X-FTN-MSGID 13127.usenet_altbbsfg@1:266/404 1f5fe07c
X-FTN-PID Synchronet 3.17a-Win32 Apr 30 2018 MSC 1800
Editor Open!EDIT
when_written 5AFA7735 C12C Tue May 15 2018 01:59:17 EDT
when_imported 5AFA7735 C12C Tue May 15 2018 01:59:17 EDT
type 0000h
version 0121h
attr 0000h
auxattr 00000000h
netattr 00000000h
header offset 298120h
header length 451
number 13127
thread_id 13127
data offset 25AB00h
data field[0] TEXT_BODY, offset 0, length 317


Again, when I upload a REP pkt and it gets imported into the sub, there's no Message-ID assigned although it shows on when I run the msgutil.js.


--

Bill

Telnet: tequilamockingbirdonline.net
Web: bbs.tequilamockingbirdonline.net
FTP: ftp.tequilamockingbirdonline.net:2121
IRC: irc.tequilamockingbirdonline.net Ports: 6661-6670 SSL: +6697
Radio: radio.tequilamockingbirdonline.net:8010/live


... Look Twice... Save a Life!!! Motorcycles are Everywhere!!!
--- MultiMail/Win32 v0.50
þ Synchronet þ TequilaMockingbird Online - Toms River, NJ

Bill McGarrity wrote to Digital Man on 05-15-18 02:24 <=-

That's exactly the message I received....

5/15 01:54:10a sub: usenet_altbbsfg, newsgroup: alt.sports.baseball.sf-giants
5/15 01:54:10a usenet_altbbsfg.ini export ptr: 13125
5/15 01:54:10a usenet_altbbsfg.ini import ptr: 331934
5/15 01:54:10a Message #13126 is missing a Message-ID header field
5/15 01:54:10a alt.sports.baseball.sf-giants import ptr: 331934, first_msg: 331859, last_msg: 331934

I noticed I gave you the wrong msgutil.js info previously...

13126
total_votes = 0
number = 13126
offset = 0
to = All
from = Bill McGarrity
subject = Another test....
from_ext = 1
to_agent = 0
from_agent = 0
replyto_agent = 0
to_net_type = 0
from_net_type = 0
replyto_net_type = 0
forwarded = 0
expiration = 0
priority = 0
cost = 0
type = 0
version = 289
attr = 0
auxattr = 0
netattr = 0
when_written_time = 1526363460
when_written_zone = -16084
when_written_zone_offset = -240
when_imported_time = 1526363535
when_imported_zone = -16084
when_imported_zone_offset = -240
thread_id = 13126
thread_back = 0
thread_next = 0
thread_first = 0
delivery_attempts = 0
last_downloaded = 0
times_downloaded = 0
data_length = 413
date = Tue, 15 May 2018 01:51:00 -0400
votes = 0
id =
5AFA758F.13126.usenet_altbbsfg@tequilamockingbirdon
line.net>
field_list =
can_read = true

Header from the operator menu..

To All
Subject Another test....
SenderExt 1
Sender Bill McGarrity
when_written 5AFA7544 C12C Tue May 15 2018 01:51:00 EDT
when_imported 5AFA758F C12C Tue May 15 2018 01:52:15 EDT
type 0000h
version 0121h
attr 0000h
auxattr 00000000h
netattr 00000000h
header offset 298020h
header length 140
number 13126
thread_id 13126
data offset 25A900h
data field[0] TEXT_BODY, offset 0, length 386
data field[1] TEXT_TAIL, offset 386, length 27


Hope this helps..


--

Bill

Telnet: tequilamockingbirdonline.net
Web: bbs.tequilamockingbirdonline.net
FTP: ftp.tequilamockingbirdonline.net:2121
IRC: irc.tequilamockingbirdonline.net Ports: 6661-6670 SSL: +6697
Radio: radio.tequilamockingbirdonline.net:8010/live


... Look Twice... Save a Life!!! Motorcycles are Everywhere!!!
--- MultiMail/Win32 v0.50
þ Synchronet þ TequilaMockingbird Online - Toms River, NJ

Re: exec/newslink.js
By: Bill McGarrity to Digital Man on Tue May 15 2018 02:24 am

Again, when I upload a REP pkt and it gets imported into the sub, there's no Message-ID assigned although it shows on when I run the msgutil.js.

Yeah, that part was expected and not a bug. The other part (where the "id" property was not being dynamically generated for the message header) should now be fixed in CVS.

digital man

Synchronet/BBS Terminology Definition #15:
DOS = Disk Operating System (as in PC-DOS and MS-DOS)
Norco, CA WX: 70.7øF, 52.0% humidity, 13 mph ENE wind, 0.00 inches rain/24hrs

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

Digital Man wrote to Bill McGarrity on 05-15-18 16:39 <=-

Re: exec/newslink.js
By: Bill McGarrity to Digital Man on Tue May 15 2018 02:24 am

Again, when I upload a REP pkt and it gets imported into the sub, there's no Message-ID assigned although it shows on when I run the msgutil.js.

Yeah, that part was expected and not a bug. The other part (where the
"id" property was not being dynamically generated for the message
header) should now be fixed in CVS.

OK.. great!!

Just read the fix... I'm amazed with your knowledge. Glad it's fixed.

Thank you..

--

Bill

Telnet: tequilamockingbirdonline.net
Web: bbs.tequilamockingbirdonline.net
FTP: ftp.tequilamockingbirdonline.net:2121
IRC: irc.tequilamockingbirdonline.net Ports: 6661-6670 SSL: +6697
Radio: radio.tequilamockingbirdonline.net:8010/live


... Look Twice... Save a Life!!! Motorcycles are Everywhere!!!
--- MultiMail/Win32 v0.50
þ Synchronet þ TequilaMockingbird Online - Toms River, NJ

exec newslink.js 1.110 1.111
Update of /cvsroot/sbbs/exec
In directory cvs:/tmp/cvs-serv19486

Modified Files:
newslink.js
Log Message:
Added an optional newslink.cfg configuration setting: reader_mode
if present in the cfg file, newslink will send an RFC 2980/3977 compliant
"MODE READER" command before authenticating with the news server. Some news servers require this command be sent by the client before supporting so-called "reader commands", like "GROUP". See https://mail.python.org/pipermail/mailman-users/1999-October/002396.html
for an example.

Thanks, EC.



---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

exec newslink.js 1.111 1.112
Update of /cvsroot/sbbs/exec
In directory cvs:/home/rswindell/sbbs/exec

Modified Files:
newslink.js
Log Message:
Added support for new configuration keyword: auto_areas
When present, all sub-boards flagged as Internet/NNTP networked will automatically be included as a newslink'd area (no "area" line needed).
If the sysop wants to override the configured/default newsgroup name,
the flags or the attachment directory for a specific sub-board/area, they
can still use the "area" keyword to specify those values for a sub-board.
Fixed the long standing UNRECOGNIZED KEYWORD: null error upon a blank
line. Apparently truncsp(null) will return "null".



---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

exec newslink.js 1.112 1.113
Update of /cvsroot/sbbs/exec
In directory cvs:/home/rswindell/sbbs/exec

Modified Files:
newslink.js
Log Message:
Strip excessive CR's.



---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/commit/064fb683a3a7eee4743dbad8
Modified Files:
exec/newslink.js
Log Message:
Twitlist optimization (don't re-read over and over)

Also added debug-level log messages for message loop rejection
reasons, for John Dovey (GATOFUEG).

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

Re: exec/newslink.js
By: Rob Swindell to Git commit to main/sbbs/master on Mon May 24 2021 21:43:53

Twitlist optimization (don't re-read over and over)
Also added debug-level log messages for message loop rejection
reasons, for John Dovey (GATOFUEG).

BTW, thank you for this. I completely reinstalled my Ubuntu box, and redid the SBBS from scratch and somewhere in that process my problem was fixed. No idea what it was now though <shrug>
---
* El Gato de Fuego * 4:92/69 (FidoNet) * Pedasi, Panama


John

... Bend the facts to fit the conclusion. It's easier that way.
---
þ Synchronet þ El Gato de Fuego - Pedasi, Panama - gatofueg.synchro.net

https://gitlab.synchro.net/main/sbbs/-/commit/ec234452f8f7c4907c9891bd
Modified Files:
exec/newslink.js
Log Message:
Add support for optiona "approved" option to be specify moderator's email addr

Apparently required by some news servers.
As requested via IRC:
<poindexter> Hi all, I've got a weird request. I'd like to post to a newsgroup that relies on automoderation. You need to add an approved: header to any posting to have it be accepted - keeps the non-techie riffraff away. is there a way to do that with newslink.js?

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/commit/20285f609d8c3d220b62cbe0
Modified Files:
exec/newslink.js
Log Message:
Merge branch 'master' of gitlab.synchro.net:main/sbbs

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/commit/38c96e07b8ec2a269b230150
Modified Files:
exec/newslink.js
Log Message:
Increase the connect timeout from 10 seconds (the default) to 30 seconds

Should probably make this a command-line option too, but this will do for
now for Ragnarok to test eternal-september with.

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/commit/483355a47172517a193a8600
Modified Files:
exec/newslink.js
Log Message:
Do not log "0 areas"

area.length is an object, not an array, so it's length will always be 0

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/commit/3d56ee8bb20e5d97ddc46c6e
Modified Files:
exec/newslink.js
Log Message:
Don't report how many bases are being scanned, cause we don't know

... was always reporting 0 because 'area' is not an array object

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/commit/c0feb81856ad898a033d903c
Modified Files:
exec/newslink.js
Log Message:
Return 1 on failure (e.g. authentication failure), not 0

Only log message header read error if the msgbase.status is non-zero

Don't try to import anything if the stop semfile is signaled while
exporting.

Increment revision to 1.2

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/commit/f55339bfecced0b573774f63
Modified Files:
exec/newslink.js
Log Message:
Don't download more msgs than the max for the sub-board (if non-zero)

No sense downloading articles we're just going to toss out the next time
we performance msgbase maintenance.

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/commit/f87b056d4236058acb30a82d
Modified Files:
exec/newslink.js
Log Message:
Print the correct .ini filename where ptrs are saved

was printing the filename in uppercase if configured as such

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net